Migrating an existing App to Heroku Celadon Cedar Stack

It’s currently not possible to do an automated migration from or to the Heroku Celadon Cedar Stack which started in May. The only help that you get from Heroku is this:

Migrating From Bamboo to Cedar

Before migrating to Bamboo, you should make sure your app runs on Ruby 1.9.2. If your app is not already deployed successfully to bamboo-mri-1.9.2, you should do that first, and then come back and try Cedar.

Once you’re sure your app is fully compatible with Ruby 1.9.2, create an app on the Cedar stack and push to it. Currently, stack:migrate does not work for moving apps to Cedar.

Since there seem to be no blogposts of how to do all the stuff with an existing application I will try to cover everything that I did to do the migration manually in this article.

Cedar is different

If you are looking at the new stack you will notice that a lot has changed with Cedar. Most of the Heroku CLI has changed to reflect the new possibilites that come with the Cedar stack. Since Cedar introduces a new way of handling processes on Heroku, most of the commands are now streamlined with this:

# old
heroku console
heroku rake db:migrate
# new 
heroku run console
heroku run rake db:migrate

# have a look at running processes
heroku ps

# look at logs of different processes and tail the output
heroku logs --ps web -t

Preparing the Migration

I started by cloning my current app from Heroku into another directory and creating a new Cedar app on Heroku:

git clone [email protected]:hamburg-on-ruby.git legacy
cd legacy
heroku create --stack cedar

Adapting Cedar changes

The docs for creating a Rails 3 app on Cedar reflect some of the changes that have been introduced with Cedar.

One of those is that PostgreSQL is now recommended for local development and you need to have the pg-gem in your Gemfile. Otherwise you will get errors while running rake commands:

heroku run rake db:migrate

  Running rake db:migrate attached to terminal... up, run.1
  (in /app)
  rake aborted!
  no such file to load -- pg

Since I developed my app with SQLite3 and I don’t want to learn, install and manage ANOTHER Database I split up the database stuff for development, test and production:

# Gemfile
group :production do
  gem "pg"
end

group :development, :test do
  gem "sqlite3-ruby", :require => "sqlite3"
end

and exclude production dependencies from my local environment:

# put this into your .profile
alias bd="bundle --without=production"

It’s also recommended to serve Rails through thin webserver and manage the application with the Procfile manager foreman:

# Gemfile
gem "foreman"
gem "thin"
# Procfile
web: bundle exec rails server thin -p $PORT
foreman start

After going through those chages you should be able to see your application on localhost:

open http://localhost:5000

The migration process

If the application is running on your development machine you can start the core migration process by putting your legacy application into maintenance mode, so that no database changes will be made:

heroku maintenance:on
# stop worker, crons or whatever might be changing the db

Database Migration

After you completely halted everything on your dyno, you can start dumping the database to your development machine (I thought it would be neat to do a direct migration from the legacy database to the new Cedar database, but I did not get it working). You need to install the taps-gem in order to do any dumping. Taps will dump the database into your development database unless you provide an connection string to another database:

# use sqlite for dumping and do it into a separate file
heroku db:pull sqlite://dump.db

# mv the file to your Cedar application and push it
heroku db:push sqlite://dump.db

Config variables

If you have custom config variables in your application it’s pretty easy to move them into the new environment:

# list all config values of the legacy app in console format
heroku config -s

# replace the newlines with whitespaces and
# append all the configuration variables of 
# your new Cedar app at the end or remove the
# ones that you do not want to migrate!

# add the list of values to your Cedar app
heroku config:add KEY=VALUE KEY2=VALUE2 [...]

Installing addons

It’s also most likely that you want to use the same addons than in your legacy application:

# list your addons in the legacy app
heroku addons

# install all addons separately in the Cedar app
heroku addons:add apigee:basic
heroku addons:add cron:daily
[...]

Sendgrid

If you are using the Sendgrid Addon you should be aware that the automatic configuration is not working anymore the Cedar Stack! You need to configure it in your app:

  # config/environments/production.rb
  config.action_mailer.raise_delivery_errors = true
  config.action_mailer.smtp_settings = {
    :address        => "smtp.sendgrid.net",
    :port           => "25",
    :authentication => :plain,
    :user_name      => ENV['SENDGRID_USERNAME'],
    :password       => ENV['SENDGRID_PASSWORD'],
    :domain         => ENV['SENDGRID_DOMAIN']
  }

Switching domains

Before you do the last step, you should have a look if everything is up and running:

heroku open

If everything is fine, you can move all the domains from the legacy application to your new and shiny Cedar Stack application:

# list domains in legacy app
heroku domains

# and remove them
heroku domains:remove hamburg.onruby.de
heroku domains:remove onruby.de
[...]

# re-add them to Cedar application
heroku domains:add hamburg.onruby.de
heroku domains:add onruby.de
[...]

Looking at the result is quite disappointing, because nothing has changed for the user, but everything is new and shiny from the inside!

If you have any other pointers of what you need to do for migrating your application please feel free to add a comment!

Developing ANDROID sucks!

DISCLAIMER: you guessed it, this is a rant!

I am very interested in doing mobile development as a hobby. I started iOS development about a year ago but i stopped after publishing my first iPhone app dzone mobile because i thought that all those AppStore™ and iTunes Connect™ stuff was a pain in the ass and distracting me from beeing productive.

So I began developing this same app for the ANDROID platform, in order to get a feeling for it. I’ve been doing Java for about 7 years now, so developing on ANDROID should be rather familiar to me instead of learning all this verbose Objective-C nerdy pointer stuff. I am very well known to Eclipse, so working with the ANDROID Plugin should be easy, right?

I started the app in November of 2010, when I had some spare time between jobs, but somehow I stopped in the middle of the project and just recently started all over. I was wondering why I had not worked on the app for about half a year?

The answer is simple: Developing ANDROID sucks!

Eclipse

I stopped working with Java in the middle of last year, switching to Ruby development with TextMate and MacVim. Coming back to Eclipse is like getting hit by a truck! This IDE is so overwhelming, but instead of giving you what you need it hides the most common features, so that you have to search for it on stackoverflow.

Is it really necessary for me to do Open-Clean-Refresh-Close-Open-Refresh cycles on my project to let Eclipse realize that everything is fine besides Eclipse itself?

Why do people, building Eclipse plugins, make the same errors again and again? Swallow errors, so that you need to have the Error Log View open all the time? Show build failures without any hints? Are you serious about that LogCat stuff? WTF?

I am using the debugger all the time when I am new to some Framework in order to better understand what is going on. But this is not very usefull unless you can see the source code of where you currently are… Guess what, it’s reeeeally hard to get it going!

Emulator

The ANDROID Emulator is really slow on the Mac. I don’t know if it’s much faster on other systems, but i think it sucks. The startup time takes even more than eclipse, you need to fiddle around with runtime settings in order to adjust it to your needs and why is it so difficult to put all the options you have in the menu bar instead of hiding it somewhere in the function keys?

But the best thing is, that the emulator is just not working right in the beginning.

It does not properly respond to mouse gestures or the keyboard and ah yeah, I forgott, it’s ugly!

Desing-Flaws

Developing for iOS is a tough choice if one is coming from a language like Java that has a decent Runtime with garbage collection. I spent most of my development time with allocation/deallocation problems. This is something you really need to get into if you want to have fun working with Objective-C even though they just released ARC (automatic reference counting) on iOS 5. Besides that, I really think that iOS does a better job in designing an architecture that is developer friendly.

The thing that I just can’t get over on ANDROID devices is that the application (the activity to be precise) is going to be reloaded if you rotate your device and you need to take care of a lot of things in order to let your app respond to that interaction. Just have a look at your future self

It’s really sad to me, that the ANDROID designers decided to repeat the bad design decisions that had been established in Java Enterprise™ all over again! It took the Java community about a decade to step back and come to frameworks where developer productivity is THE thing to focus on. All the rookies that start their “Java career” with ANDROID will be doomed to go through this same process again! Poor bastards…

XML

Finally! Let’s define everything in XML! I am getting bored at bashing on XML…

$ find . -name "*.xml"
./AndroidManifest.xml
./res/layout/details.xml
./res/menu/options.xml
./res/values/strings.xml
./res/xml/preferences.xml

iOS is also using XML for their interfaces, but they hide all that shit behind a really decent InterfaceBuilder!

The Builder and XML Tools in the Eclipse Plugin are so ugly that i rather write the whole shit by hand, BITCHES!

Tight coupling

Why, for gods sake, did they come up with the idea to use inheritance for everything in ANDROID? It’s even worse than in J2EE because you are not bound to Interfaces but Classes! Hello dear ANDROID designers, there is no such thing as Mixins or Multi-Inheritance in Java?! How am I supposed to write clean code with that? Answer: You are not going to write clean code anyway, because you have to do premature optimization on your code so that it can run on all the shitty ANDROID devices! YEAH!

Just forget everything you learned form state of the art frameworks like MVC or DI. It’s just not possible! BY DESIGN!

Just put state, lifecycle, business and view logic in one class; also helps reducing your package size, HOORAY!

Checked Exceptions

Do we really need more Checked Exceptions? Really?

Testing

Writing tests for iOS was nearly impossible at the time I was doing it. Even the guys at the Cocoaheads Hamburg did not know how to get it running. But this situation has improved over the last months. Special thanks to XING for doing a lot of research there and presenting it on the Usergroup!

I thought that testing for ANDROID would be really cool, because the Java community has a really strong background on TDD and Testing is even a Topic on the Dev Guide, even though it is the LAST one…

Let me make it clear to you: Testing ANDROID sucks!

Because of all the tight coupling and multithreaded environment it is nearly impossible to test something. Even simple tasks like testing your Utility-Methods needs to be done in the Emulator! Just have a look at this multiscreen setup guide of how to get it running with JUnit (Hello?! JUnit 3 is deprecated for YEARS!).

Despite all this crap, I finally did it! So please test my dzone application, because I do not own an ANDROID phone myself…

AWS cart operations support in ASIN

It has been a little silent in here lately. This is probably due to my new freelance work as a Rails developer at Qype.com in Hamburg…

Nevertheless, I am proud to announce, that, thx to Rehanift’s fork, ASIN now support AWS cart operations:

  #just require
  require 'asin'

  # create an ASIN client
  client = ASIN.client

  # create a cart with an item
  cart = client.create_cart({:asin => '1430218150', :quantity => 1})
  cart.items
  => [<#Hashie::Mash ASIN="1430218150" CartItemId="U3G241HVLLB8N6" ... >]

  # get an already existing cart from a CartId and HMAC
  cart = client.get_cart('176-9182855-2326919', 'KgeVCA0YJTbuN/7Ibakrk/KnHWA=')
  cart.empty?
  => false

  # clear everything from the cart
  cart = client.clear_cart(cart)
  cart.empty?
  => true

  # add items to the cart
  cart = client.add_items(cart, {:asin => '1430216263', :quantity => 2})
  cart.empty?
  => false

  # update items in the cart
  cart = client.update_items(cart, {:cart_item_id => cart.items.first.CartItemId, :action => :SaveForLater}, {:cart_item_id => cart.items.first.CartItemId, :quantity => 7})
  cart.saved_items
  => [<#Hashie::Mash ASIN="1430218150" CartItemId="U3G241HVLLB8N6" ... >]

You can use the latest source from GitHub development branch or download the beta version of the gem from RubyGems. No easter eggs included :)

Happy shopping(cart)!

Gem Best Practice

One thing that I like most about Ruby is the ease of contributing to OpenSource. You want to share code? Just create a gem and upload it to Rubygems. The problem is, there are a lot of implicit rules that make it hard to do it right in the first place.

Enabling Rubygems

Since Ruby 1.9 Rubygems is bundled with every Ruby distribution. If you are using RVM you get Rubygems with the older versions too. Setting the RUBYOPT environment variable will enforce it even for 1.8 versions:

# no need for 'require "rubygems"' in your code
export RUBYOPT=rubygems 

The Gem command

Since the merge of Gemcutter and Rubygems there have been a lot of improvements to the gem shell command. Sharing your code is now as easy as:

# create a gem from your local source configured in your gemspec
gem build your.gemspec
# upload the created gem 
gem push your-0.0.1.gem

There are some more new commands that handle the Rubygems API communication, which are very well documented on RubyGems.org.
It is also worth taking some time to get an overview of the old command reference. Commands like which or cleanup come in handy if you know them!

I use RVM and Gemsets extensively and I do not want to have all that duplicated RDoc and RI on disc (I’d rather search the web instead of using local generated doc), so I changed my .gemrc file to skip generating the doc on gem install:

# .gemrc
--- 
:update_sources: true
:sources: 
- http://gems.rubyforge.org/
- http://gems.github.com
gem: --no-ri --no-rdoc
:verbose: true
:bulk_threshold: 1000
:backtrace: false
:benchmark: false

Building a Gem

Jeweler is a great tool for scaffolding and releasing new Gems. If you don’t care about getting to know the basics, just move right over to their documentation. The problem is, that it makes things more complicated than they really are…

Naming and Versioning

There are some simple rules when it comes to naming:

  1. Use lowercase letters only! Users of case insensitive file systems will be thankfull.
  2. Use ASCII characters only! You know, those US guys just don’t know there is UTF-8.
  3. Use _ underscores for parts of the name and - dashes for extensions of existing Gems.
  4. Name your Gem like the file to require in your code. If not, put a big, fat warning in your documentation!

Choose the Gem name with care, because you know: naming is everything!

It became popular lately to start the development cycle with a version 0.0.1 instead of 1.0.0. I think this is just a matter of taste. More importantly you should handle your MAJOR.MINOR.FIX versions carefully. Increment your FIX version on bugfixes or additions to the API that won’t brake any existing code. Increment your MINOR version if you added features with an noticable impact to the API or it’s usage. Push a new MAJOR version if you changed your API completely or if you want to emphasize that the latest version is a major improvement or a new stable release. In an ideal world all those decisions and changes are documented in a CHANGELOG file.

In addition to these rules, Rubygems provides a mechanism to roll out beta versions of a Gem. In order to do this, just append a .beta etc. to the end of your version. These Gems can then be installed with the —pre flag:

gem install rails --pre # might install rails 3.1.0.beta some day

The Gemspec

One thing that is essential for creating a Gem is the .gemspec file. Put one in the root of your project folder and name it alike. The .gemspec file contains plain old Ruby and simply specifies all important attributes for your Gem. For an example, have a look at the gem_best_practice.gemspec:

# coding: utf-8
lib = File.expand_path('../lib/', __FILE__)
$:.unshift lib unless $:.include?(lib)

require 'gem_best_practice'

spec = Gem::Specification.new do |s|
  s.name = s.rubyforge_project = 'gem_best_practice'
  s.version = GemBestPractice::VERSION

  s.author = 'Peter Schröder'
  s.description = 'A Gem that shows best practice for Gem creation.'
  s.email = '[email protected]'
  s.homepage = 'http://github.com/phoet/gem_best_practice'
  s.summary = 'If you ever wondered how to create a Gem and what best practice should be applied, this project is for you.'

  s.has_rdoc = true
  s.rdoc_options = ['-a', '--inline-source', '--charset=UTF-8']
  
  s.files = Dir.glob('lib/**/*.rb') + Dir.glob('bin/*') + %w(README.rdoc)
  s.test_files = Dir.glob('test/**/*.rb')
  
  s.executables = %w(gbp)
  s.default_executable = 'gbp'
  
  s.add_dependency("erubis", "~> 2.6.0")
end

After the .gemspec file is done, the Gem may be built via the gem command:

gem build your.gemspec

Or with a custom Rake task:

spec = eval(File.new("your.gemspec").readlines.join("\n"))
Rake::GemPackageTask.new(spec) do |pkg|
  pkg.need_zip = true
  pkg.need_tar = true
end

Folder Structure

Ruby is an open language with very little restrictions. You can place your code anywhere, but there are some guidelines of how code should be structured so that nobody has to mess with the $LOAD_PATH in order to use it. This is an example of standard Gem folders:

gem_best_practice/
|- bin
|- example
|- lib
|- test

Your actual Ruby code should go under lib and subfolders. It’s a best practice to put just one file there and the other stuff right into another subfolder in order do avoid name clashes or the like. Extensions to the standard Ruby libraries should be separated out, so that they are easier to spot:

|- lib
|   |- gem_best_practice
|   |   |- core_ext
|   |   |   |- kernel.rb
|   |   |- core_ext.rb
|   |   |- gbp.rb
|   |- gem_best_practice.rb

All the tests should go into the test folder ( spec / feature folder respectively). Make sure the Gem is well tested, otherwise chances are nobody will use the it nor contribute to the codebase. As an awesome Ruby developer, one would add some (working) examples in the example folder to demonstrate the API of your Gem in action. Additional executables for your Gem should be put into bin and don’t forget to add a proper shebang:

#!/usr/bin/env ruby

Dependencies

There are a lot of great libraries available for Ruby nowadays and one would be a fool not to use them. That’s why most of the Gems depend on other libraries. Different Gems may share common dependencies, so it’s elementary for a good Gem to be as loose about the own dependencies as possible to reduce versioning conflicts. Tools like Bundler do a great job on resolving those conflicts and they depend on the versions that are wired in the .gemspec file of a Gem. You can easily tell Bundler to use your .gemspec the main configuration source for your gems. A tradeoff between hard wiring Gem versions and providing no version restrictions at all, is to use the ~> range operator, that will allow newer dependency versions within the same MINOR version:

  s.add_dependency("erubis", "~> 2.6.0") # >= 2.6.0 and < 2.7.0

There is still one problem left. It’s currently not possible to define optional dependencies in your .gemspec file. This is a big problem for libraries that implement some kind of adapter pattern (Have a look at HTTPI which provides an interface to different HTTP libraries) and don’t want to force the user to install all underlying implementations (This is also a problem for bundler).

Until this problem is fixed in Rubygems, a simple approach to this problem is to programmatically rescue from LoadErrors:

# require optional dependency
begin
  require 'ap'
rescue LoadError
  puts "awesome_print would be awesome, please 'gem install awesome_print'"
end

Documentation and Examples

An awesome Gem deserves a decent documentation. If you are too lazy to document your code, you should at least bundle a README file with your source. GitHub does a great job at pushing a project to provide a README (rdoc/textile/markdown/…), because this file will be prominently rendered on the projects start page. The README should at least include installation instructions and basic usage examples.

Using YARD / RDoc in the source code is another great way to help others understand the code, provide bugfixes or extend existing functionality. Even for yourself, having a good inline documentiont will save time when returning to a piece of code you did not had your hands on for month.

Generating the documentation can easily be done with Rake:

Rake::RDocTask.new(:rdoc_dev) do |rd|
  rd.rdoc_files.include("lib/**/*.rb", "README.rdoc")
  rd.options + ['-a', '--inline-source', '--charset=UTF-8']
end

If the project is hosted on GitHub, rdoc.info may generate the documentation automatically from the source-repository. GitHub also provides a service-hook so that the documentation is updated on every push! A .document file in the project will give rdoc.info the right pointers which files to include in your docs:

lib/*.rb
README.rdoc

Depending on the project, it might be a good idea to include some usage examples with the code. This might be the case if one can not put all the necessary information in the inline comments or the README file. Bye the way, (especially behavior driven) tests serve very well as executable examples.

Another thing that is overlooked quite often is the documentation of Ruby runtime compatibility. Does the Gem work with MRI 1.8 / 1.9 / JRuby / REE / … ? Since RVM is a defacto standard in the Ruby environment, there is no good reason not to test at least Ruby 1.8.7 and 1.9.2.

And then?

After finishing the work on the first version of a Gem and pushing it to Rubygems, what comes next?

If you are just interested in the acceptence of the Gem you can have a look at the Rubygems user profile or individual Gem stats.

A good way to help the Gem gain acceptance is to make it OpenSource via GitHub or RubyForge.

It’s not easy to reach a wide range of users, but if you want to promote your Gem over the interwebs there are a lot of services like DZone or Forrst that allow you to post something about the Gem or to link to articles on your blog or Twitter posts. Another great resource for Ruby developers to discover new Gems is The Ruby Toolbox.

The Gem Best Practice Gem

I am currently putting all those Guidelines above into a Gem Best Practice Gem Project on GitHub which is inspired by bundler’s bundle gem command.

More best practice, less fail!

Updating Savon Handsoap Shootout

Since my initial series of blog posts about Savon and Handsoap a lot has been done on the Savon side.

About a week ago rubiii has released the first beta of Savon 0.8. A lot of bugfixes, improvement and API changes have been addressed in the latest version. If you want to get used to the new API a good starting point is the Savon Guide.

One thing that should be noted right away, is that Savon 0.8 is not downwards compatible. Most code parts have been rewritten and Savon now offers an improved error handling, support for local wsdl documents and more flexibility on request methods. See the changelog for more information.

Shootout update

I have updated the examples used for the shootout to reflect the latest changes for Savon 0.8.beta3, Handsoap 1.1.7 and SOAP4R. I restructured the examples, removed dead services and added Bundler for gem dependencies. The examples run on MRI 1.8.7 and JRuby 1.5.2.

More Savon infrastructure

Some effort has been taken to make the day to day work with Savon even more pleasant. If you are working with RSpec, you can now test using Savon Spec.

If you are used to work with ActiveRecord models Savon Model will help integrating SOAP into your Rails projects.

SOAP without fail!