Tag: Redis

Heroku Cedar Background Jobs for free!

I’m using Heroku mostly for playing around with latest technology and hosting free apps. Since Heroku changed their pricing model due to the introduction of the new process model some of my apps changed from free to paid, especially those that had some background jobs or nightly crons (I really did not get, why this happend).

Full Stack Background Processes

If you want to run a resque worker and a clockwork process within your web-app, this becomes a costly thing, even if those are just running some minor jobs in the night, because you need to pay for 2 additional dynos.
You could achieve this through defining multiple processes in your Procfile:

# Procfile
web: bundle exec rails server thin -p $PORT
worker: QUEUE=* bundle exec rake environment resque:work
clock: bundle exec clockwork app/clock.rb

heroku scale worker=1 clock=1

Sharing Addon Connections

As Heroku officially announced in their latest newsletter, it’s possible to share connections between multiple apps. In my case, this would be a connection to a redis key-value store, that is provided by Redistogo. It’s as simple as copying the environment configuration for the addon over to the second app:

heroku config | grep DATABASE_URL  --app sushi
=> DATABASE_URL   => postgres://lswlm...

heroku config:add DATABASE_URL=postgres://lswlm... --app sushi-analytics
=> Adding config vars: DATABASE_URL => postgres://lswlm...m/ldfoiusfsf
=> Restarting app... done, v74.

Going Freemium

So the solution for getting back to a free worker setup is combining 3 Heroku apps and a shared Redis connection through Redistogo:

heroku apps:create freemium-web --stack cedar --remote heroku
git push heroku master

heroku apps:create freemium-worker --stack cedar --remote worker
git push worker master

heroku apps:create freemium-clock --stack cedar --remote clock
git push clock master

heroku scale web=1 --app=freemium-web
heroku scale web=0 worker=1 --app=freemium-worker
heroku scale web=0 clock=1 --app=freemium-clock

heroku addons:add redistogo:nano --app=freemium-web

heroku config:add `heroku config -s --app=freemium-web|grep redis` --app=freemium-worker
heroku config:add `heroku config -s --app=freemium-web|grep redis` --app=freemium-clock

I created an example project running a Rails 3 application with a mounted Resque web, a Resque worker and a clock process with Clockwork.

Even simpler

If you are just looking for a simple solution of running a background process have a look at crony, a bootstrap project using rufus-scheduler.

Using the Redis addon on Heroku

I am always playing around with new addons offered by Heroku. My latest discovery was the Redis addon that is provided by Redistogo. The addon is probably in private beta (docs are still on beta), but since they put up a link to it on their site, I managed to install it to my personal website that runs in the cloud.

Redis is “an advanced key-value store” and has some features that make it a perfect match for a cache! I use caching extensively on my site and keep on trying out new ways to do it to circumvent Heroku’s readonly filesystem.

Like Memcache, Redis provides the ability to set a time to live (ttl) on a key. This comes in handy, if you have data that expires in a short period of time, like 3rd party data from Twitter etc.

Caching with Redis

Accessing Redis is very simple, since it is a text based protocol. The command reference is straight forward and there is a simple Ruby wrapper available:

require "redis"
redis = Redis.new
redis.set "foo", "bar"
# => "OK"
redis.get "foo"
# => "bar"

The redis-store gem already provides a Rails 3 compatible Cache Store implementation, but I needed some more configuration points, especially the ttl.

That’s why I wrote my own Rails 3 Redis Cache, also a great way to get used to the way of working with Redis and the Redistogo addon.

Using Rails Redis Cache

There is some configuration needed for Rails to pick up the new cache store. If you want to use different or no caching for test, development and production, you should put the config in your environment files:

# config/environemnts/production.rb
config.action_controller.perform_caching = true
config.cache_store = ActiveSupport::Cache::RailsRedisCache.new(:url => ENV['REDISTOGO_URL'])

If there is a Redis server available in all environments, you can put it in your environment file:

# config/environment.rb
ActionController::Base.cache_store = ActiveSupport::Cache::RailsRedisCache.new(:url => ENV['REDISTOGO_URL'])

The caching parts are mostly in my controllers:

@tweets = cache("tweets", :expires_in => 30.seconds){ Twitter::Search.new(...) }

The store is using the basic Rails cache store implementation which is broken in the Rails 3.0.0.beta1 version that runs on Heroku, so I added a monkey-patch for that using edge Rails.

Redis on localhost

Installing and running Redis on Mac OS X is really simple:

brew install redis
redis-server

There is also a commandline client available for direct access:

redis-cli
redis> set "foo" "bar"
OK
redis> get "foo"
"bar"

It’s key value stores, stupid!