Swivel Code Blog

Changing its spots ... OrderedHash chaos in Leopard

UPDATE: The brilliant people at MacPorts have fixed the bug we encounter below. To upgrade:

svn checkout http://svn.macports.org/repository/macports/trunk/dports/lang/ruby/
cd ruby
sudo port install

---

Sometimes, you find yourself using hashes as keys in a hash. It gets meta:

Item.all.group_by{ |i| i[:attrs] }

Where attrs is a hash like this:

{ :id => Fixnum, :type => String }

After a recent upgrade of some vendor code, some of us starting seeing test failures around that group_by code, and some didn't, although we were all on the same ruby version and patchlevel. The tests were still passing on the build server. And the failures made no sense -- the values of the hash were nil.

After some head-scratching, we found that Ruby running on Mac OS Leopard can't handle hashes with large numbers as values very well.

>> {:a => 536870911}.hash == {:a => 536870911}.hash
    => true
>> {:a => 536870912}.hash == {:a => 536870912}.hash
    => false

And "large numbers" are exactly what you get as default ids for fixture-created ActiveRecord objects in Rails tests. The key being assigned to and the key being read from were identical, except they weren't.

It turns out that in the Leopard environment running MacPorts the items would never have been grouped properly, but in production everything looked fine. The upgrade to Rails we made included some enhancements to Enumerable#group_by to sort the output as an OrderedHash, and that exposed us to the bug -- now, instead of just failing to group the results properly, it was breaking. Those who had recently upgraded to Snow Leopard did not have the same problem -- and our servers, obviously, were immune.

Sending Email with ActionMailer -- New! Improved! DRYer!

In a previous post, we talked a bit about DRYing up email. You can alias all the methods for delivering different types of email at a generic method, and then simply control all the differences at the template level:

  ['feedback', 'sales', 'hate'].each { |t| alias_method t.to_sym, :message }

Of course, leaving well enough alone isn't something we do, and the idea of having to update that hard-coded list of templates doesn't seem logical. All the templates will live in the views/notifier directory, so why not just pull them from there when the app starts up?

Within the Notifier model, instead of the hard-coded line above, create a class method that calls alias_method on each element in an array:

  def self.templates= templates
    templates.each do |t|
      alias_method(t.to_sym, :message)
    end
  end

Then, create a new initializer called something like config/initializers/populate_email_templates.rb which just has this handy one-liner:

  Notifier.templates= Dir.new("#{Rails.root}/app/views/notifier").entries.map{|f| File.basename(f, '.html.erb')}

Kick the server so it picks up the changes, and you're good to go. Now when you create (or delete) an email template, all the plumbing and wiring is already done and it should just work.

Integration testing -- even your emails aren't safe

An integration-test related thought suddenly struck me: often you'll send an email that includes an action item as a link. Sometimes that link will actually do something -- confirm a change you've made, for example. Now, of course you're testing the path that the email is supposed to include, but why not go one further and test that the link that's actually in your email will have the desired effect?

[ ... send out the email .. ]
assert_change('Person.find(people(:one).id).confirmed?', 'false', 'true') do 
  assert_select_email do
    get(assert_select('a').first['href'])
  end
end

You should clear out the deliveries array first, or it might try to operate on the wrong email. I just put this line in the setup method:

ActionMailer::Base.deliveries = []

Never heard of assert_change, you say? It's a riff on assert_difference that might find a home in a test_helper.rb file near you:

  def assert_change(field, from, to)
    initial = eval(field).to_s
    assert_match(initial, from)
    yield
    assert_equal(initial.gsub(from, to), eval(field).to_s)
  end

A Quick One, While He's Away

I ran into a small problem using assert_select_email -- it expects multipart messages, and those we are sending are not.

A quick and dirty solution was to included the following ugly monkeypatch rewrite of the SelectorAssertions in our test_helper file:

module ActionController
  module Assertions
    module SelectorAssertions
      def assert_select_email(&block)
        deliveries = ActionMailer::Base.deliveries
        assert(!deliveries.empty?, "No e-mail in delivery list")
        for delivery in deliveries
          for part in delivery.parts.empty? ? [delivery] : delivery.parts.select{|p| p["Content-Type"].to_s =~ /^text\/html\W/}
            root = HTML::Document.new(part.body).root
            assert_select(root, ":root", &block)
          end
        end
      end
    end
  end
end

You And Me Baby Ain't Nothing But YAMLs

Javascript testing has come a long way from the days when it didn't exist. We've touched on the Scriptaculous unit testing framework in this space before, but the nature of a web application that's heavily AJAX-dependent is that you need to be absolutely sure the server and the client are in agreement, and for testing purposes this means that your framework tests and your Javascript tests have to be on the same page.

That page has a few requirements. It needs to be accessible from an HTML page that is accessed via the file:/// protocol -- you'll always need the option of running your tests in isolation, locally, rather than only through rake test:javascripts. It needs to be something that can be populated and maintained easily by humans. It needs to be lightweight. And it needs to be usable by both our Rails tests and our Javascripts tests. So, given the Rails paradigm of using YAML to maintain test fixtures, why not start there? Let's create an ajax_fixtures.yml file in test/javascripts and populate it with the AJAX requests and responses we want to test, as YAML data:

requests:
    save:
        value: 1
	quantity: 3
	id: 1

responses:
    save:
	result: Item successfully saved

An integration test called test/integration/ajax_test.rb is the way we roll when it comes to testing all AJAX interactions from the server perspective. In the setup method, load up the Ajax requests so they're available to every test:

def setup
    @ajax = YAML.load(File.open("#{Rails.root}/test/javascript/ajax_fixtures.yml").read)
end

Now, to test the "save" AJAX request, you can access the @ajax['requests']['save'] and see if sending that request has the desired effect:

test("saving") do
    post(item_path(items(:one), :format => 'json'), :item => @ajax['requests']['save'].to_json) 
    assert_equal(3, assigns(:item).quantity)
end

Well, that's straightforward enough and, so far as it goes, unnecessary. But let's think of that YAML file as a contract between the server and client components of the app. The Javascript tests have to prove that the right circumstances will generate the requests and handle to the responses, while the Rails tests must prove that they can interpret the requests and provide the expected responses. The Rails tests can easily handle our YAML file, but the Javascript tests might need a little help. First, we need to the YAML to HTML so that all browsers can load it. This is easily done with a rake task which you can drop into lib/tasks/somefile.rake. Choose which rake testing tasks should invoke the YAML task -- the whole test suite, obviously, and any that might be AJAX-related:

task :test => 'ajax:prepare'
namespace :test do
    [:integration, :javascript].each{ |t| task t => 'ajax:prepare' }
end

namespace :ajax do
    desc "Prepare AJAX test fixtures"
    task :prepare do
        json = YAML.load(File.open("#{Rails.root}/test/javascript/ajax_fixtures.yml")).to_json
        File.open("#{Rails.root}/test/javascript/ajax_fixtures.html", 'w').puts(json)
    end
end

This snippet will execute before you run rake or one of the specified tasks, and will write out the structured YAML to a big blob of JSON which is not very human-readable, but your Javascript test file can grab via a simple Ajax call:

  <script type="text/javascript">
    new Ajax.Request('ajax_fixtures.html', {
      asynchronous: false,
      method: 'get',
      onSuccess: function(t) {
        expectedAjax = t.responseText.evalJSON();
      }
    });
  </script>

Also in our javascript test file, we're going to call on Dr Nic for some help mocking out the Ajax requests:

 <script src="assets/drnic_js_test_helpers.js" type="text/javascript"></script>

 ...

 testSave: function() { with(this) {

   Test.Ajax.setupMock('/items/1/save.json', function(request, response) {
      ajaxRequestCount++;
      ajaxRequestJSON = request.options.parameters.evalJSON();
    }.bind(this));


   ajaxRequestCount = 0;
   ajaxRequestJSON = '';
 
   // invoke javascript 'save' method //

   assertEqual(1, ajaxRequestCount); // the Ajax Request has been called
   
   // compare the expectedAjax['requests']['save'] data (from YAML)
   // with the ajaxRequestJSON data actually passed to the Ajax.Request method

  }}

Remember to run Test.Ajax.clearMocks() when you're done --the teardown method is the best place for this. Now you've loaded the same information into your Javascript test and your Rails test and if the two agree, then you're really getting somewhere.

Special bonus section!

Comparing expectedAjax and ajaxRequestJSON can get a bit tricky if you have complicated nested objects. Here's a little object comparison method I wrote for this purpose, seen here as a standalone function. It requires Prototype:

    function hashEqual(a, b) {
      if ((!a) && (!b)) { return true;}
      if (!a.keys) { a = $H(a); } if (!b.keys) { b = $H(b);}
      if ( String(a.keys().sort()) != String(b.keys().sort())) { return false; }
      var r = a.keys().map(function(k) {
        var a_val = a.unset(k);
        var b_val = b.unset(k);
        if (typeof(a_val) == 'object') {
          return (hashEqual(a_val, b_val));
        } else if (a_val != b_val) {
          return false;
        } else { return true;}
      }.bind(this)).uniq();
      return(r.reject(function(e) { return(e); }).length == 0);
    };

Faking Keyboard Events in Safari: A Work in Progress

Usually we use this space to bring you carefully-crafted solutions to problems we've encountered, and if I had solved this particular problem that's what I'd do now. In this case, I'm not entirely sure if a solution exists or if I've come up against a browser bug, and I've exhausted the Internet's collective wisdom on the subject. So here's the state of play as regards creating keyboard events in Safari/Webkit.

One of the many great things the scriptaculous unit testing framework provides is the ability to mock events using Event.simulateKey and Event.simulateMouse.

Of course, given the state of cross-browser event handling, it should be no surprise that cross-browser event creation is a hairy beast. Scriptaculous uses the createEvent to (unsurprisingly) create events, with document.createEvent("KeyEvents");, and Safari/Webkit immediately gives this up as a bad job with NOT_SUPPORTED_ERR: DOM Exception 9. More acceptable is document.createEvent("KeyboardEvent"), which will work in most browsers.

Now that we have an event, we pass initKeyEvent with a long list of options including keyCode and charCode, the parameter so great they named it twice and implemented it slightly differently and caused enormous amounts of confusion. Safari/Webkit, though, has no time for this sort of carry-on and requires the W3C standard initKeyboardEvent, which takes a different list of different options. The character codes have been replaced with a keyIdentifier and keyLocation instead -- the first being the Unicode for the key being pressed (or the string "undefined") and the second specifying which key when several keys have the same value (a number on the numeric keypad versus its normal number key, for example). Finally, instead of booleans for the shift, alt, control, and meta key states, as initKeyEvent requires, initKeyboardEvent takes a modifiers argument, which is a string of whitespace-delimited names. The event itself will still have the keyCode and charCode attributes so your event handler code will work, but these are readonly and cannot be passed as arguments.

So, my tentative reworking of event.simulateKey might look a bit like this:

Event.simulateKey = function(element, eventName) {
  var options = Object.extend({
    ctrlKey: false,
    altKey: false,
    shiftKey: false,
    metaKey: false,
    keyCode: 0,
    charCode: 0
  }, arguments[2] || {});
  var oEvent = document.createEvent("KeyboardEvent");
  try {
  oEvent.initKeyEvent(eventName, true, true, window, 
    options.ctrlKey, options.altKey, options.shiftKey, options.metaKey,
    options.keyCode, options.charCode );
  } catch(e) {
    var modifiers = [];
    if (options.shiftKey) { modifiers.push('Shift'); }
    if (options.ctrlKey) { modifiers.push('Control'); }
    if (options.altKey) { modifiers.push('Alt');}
    var keyLocation = '0x00';
    var keyIdentifier = 'U+00'+options.keyCode.toString(16);
    oEvent.initKeyboardEvent(eventName, true, true, window, keyIdentifier,
keyLocation, modifiers.join(' '));
  }
    $(element).dispatchEvent(oEvent);

So, I've done what I can to placate the Safari gods and create a keyboardEvent according to spec -- but it seems that whatever you do, you simply cannot get Safari to create a keyboard event that truly emulates a real system keyboard event.

The keyCode and charCode attributes are stubbornly reset to 0 and the goal of running our entire test suite on Apple browsers remains. What results have other folks had, and what alternatives exist?

Sharp threads

When you come to a .fork in a Process, take it.

A foible of Ruby forked processes is that children have copies of the parent process's objects. In the case of the ActiveRecord::Base.connection_pool, this means that usually when you try to start new database connections within a forked process, you won't.

When a complex database migration requires a lot of horsepower and raw SQL executes, rather than allowing ActiveRecord to take its course, you will likely benefit from forking the task. But you may also want to ensure that your database connections are unique to each process.

Much of the advice out there focuses on how important it is to teardown the existing connection before forking, have each process create its own connection, and then reset the old connection at the end. And this is good as far as it goes, but it's not the whole picture. If you simply remove the connection, and then establish a new connection in each process, the connection_pool is going to look the same to each process and thus the "new" connection each time will actually be the same one:

  ActiveRecord::Base.remove_connection
  4.times do
    Process.fork do
      ActiveRecord::Base.establish_connection
      puts connection
      ActiveRecord::Base.remove_connection
    end
  end
  Process.waitall
  ActiveRecord::Base.establish_connection

outputs:

  #<ActiveRecord::ConnectionAdapters::MysqlAdapter:0x335e154>
  #<ActiveRecord::ConnectionAdapters::MysqlAdapter:0x335e154>
  #<ActiveRecord::ConnectionAdapters::MysqlAdapter:0x335e154>
  #<ActiveRecord::ConnectionAdapters::MysqlAdapter:0x335e154>

Not what we want. Now, there are, most likely, fancier and better ways of doing this, and it seems a little bit sketchy to have to roll one's own connection pool ... but I like things that work and are easy, so:

  ActiveRecord::Base.remove_connection
  pool = []
  4.times do |i|
    pool.push(ActiveRecord::Base.establish_connection)
    Process.fork do
      connection = pool[i].connection
      puts connection
      ActiveRecord::Base.remove_connection
    end
  end
  Process.waitall
  ActiveRecord::Base.establish_connection

And now we get:

  #<ActiveRecord::ConnectionAdapters::MysqlAdapter:0x335ea28>
  #<ActiveRecord::ConnectionAdapters::MysqlAdapter:0x3351ddc>
  #<ActiveRecord::ConnectionAdapters::MysqlAdapter:0x3345140>
  #<ActiveRecord::ConnectionAdapters::MysqlAdapter:0x33384f4>

Perfect. And, yes, those connection objects are perfectly good and you can go ahead and call connection.execute, connection.select_value, etc. exactly as you normally would.

Packaging Dojo with Capistrano

While we love The Dojo Toolkit, we're not sure if it's even possible for one application to use all of its functionality. At Swivel, we use many of dojo's charting components, but not much else. One of our chart pages will pull in several dozen Javascript files as dojo imports the resources it needs at runtime. Surely there has a way of getting all the dojo we need, and nothing else? Well, the Dojo Foundation hivemind has thought of that.

63% Extra, Free!

Let's put our results above the fold: packaging our dojo files cut the load time for our most dojo-intensive pages by 63% on average and dramatically reduced the number of individual requests. The change for one page was as follows:

There's good documentation for using Dojo's system for packaging custom builds. You'll need the src build which includes the util directory -- for our purposes we needed to tweak our .braids file to include:

public/javascripts/dojotoolkit/util:
  squashed: true
  url: http://svn.dojotoolkit.org/src/util/trunk
  type: svn
  revision: 18839
  remote: braid/public/javascripts/dojotoolkit/util

(I'll save all the gory details of installing git+svn to get braid to work locally for another, longer, and more painful post. Word to the wise: you may need the apr Portfile.)

Selecting the dojo components you need is straightforward enough. Have a look at what dojo files your page is requesting and work from there. Our build profile ended up looking like this:

dependencies = {
  layers:  [{
    name: "swivel_dojo.js",
      dependencies: [
        "dojox.charting.Chart2D",
        "dojox.charting.themes",
        'dojox.gfx.svg',
        'dojox.nls'
      ]
    }
  ],
  prefixes: [
    [ "dijit", "../dijit" ],
    [ "dojox", "../dojox" ]
  ]
};

What we wanted to do, though, was to have the raw dojo files under source control through Braid, and only use the package in production. Debugging the single humongous line of generated Javascript is tricky, for one thing, and managing the versioning of the Dojo toolkit files properly was also important. This required a bit of trickery.

Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo

We created a dojo_release directory as a sibling of dojotoolkit, and added a symbolic link called dojo that points back up to dojotoolkit. So public/javascripts/dojo_release/dojo is a symbolic link to public/javascripts/dojotoolkit. That's the version that gets checked in to source control, and all files that use our dojo build have this line of Javascript:

 <%= javascript_include_tag('dojo_release/dojo/dojo/dojo', 'dojo_release/dojo/dojo/swivel_dojo', '...' %>

Now everything should work as before, we're just getting to the dojo files in a different way. So anyone checking out the source and working with it will see no difference except the paths for those Javascript files. Or, to put it another way, we haven't really done anything yet.

What we want the dojo packaging script to do is run against our new profile and output the packaged version into the dojo_release directory, clobbering the symlink with the compiled package. The best time for this to happen would be after the code is updated via our deploy:update_code task. And, so, a new task is added to configuration/deploy.rb:

  namespace :dojo do
    desc "package dojo files using Swivel profile"
    task :package, :roles => :db, :only => { :primary => true } do
      run <<-SCRIPT
        rm #{release_path}/public/javascripts/dojo_release/dojo &&
        cd #{release_path}/public/javascripts/dojotoolkit/util/buildscripts &&
        ./build.sh profile=swivel action=release releaseDir=../../../dojo_release
      SCRIPT
    end
  end

and

    after "deploy:update_code", "deploy:dojo:package"

The SCRIPT heredoc consists of multiple lines chained with && because Capistrano will otherwise instantiate a new session for each command. It's important to go to the buildscripts directory unless you've done some munging of your Java classpaths or whatever it is Java needs, I don't know and I'm not sure I want to know. And, finally, setting the build script's releaseDir will put the packaged dojo javascript where your existing javascript_include_tag can find it.

And that's it. Run cap deploy and the script should run, packaging all the dojo code it needs into a single (huge!) line of Javascript. When you view your deployed app using a debugger like Firebug or Webkit's built-in developer tools, you'll see how the Javascript files are being served differently. And your users won't need those tools to see the difference.

Rails auto_link and certain query strings

Our local newspaper, the San Francisco Chronicle, is occasionally a source for data stories posted to Swivel, and that's how we discovered that there are a couple of edge cases that Rails's auto_link doesn't handle perfectly.

A typical Chronicle link looks something like this: <a href="http://www.sfgate.com/cgi-bin/article.cgi?f=/c/a/2007/08/24/WIJKRIC6E.DTL">. Nothing too peculiar there. However, when passed through the auto_link function, the / character at the beginning of the value of the query string variable causes the entire value to be dropped, leaving us with lots of links to <a href="http://www.sfgate.com/cgi-bin/article.cgi?f=">.

This bug has been noted by Satya among others, and the monkey-patch solution is fairly straightforward. The first step is to write a functional test:

  test("URL querystring variables can begin with a slash") do
    url = 'www.example.com?url=/rails/cannot/deal/with/this/'
    post(:create, :resource => { :name => "test", :link => url })
    get(:show, :id => assigns(:resource).id)
    assert_select("a[href=http://#{url}]")
  end

Our test fails. The URL is truncated at the =.

ActionView's TextHelper includes a regular expression constant AUTO_LINK_RE which tells Rails how to recognize URLs. We prefer not to tinker with the actual vendor code itself to avoid any upgrading gotchas or other weirdness, so I copied the relevant RegEx to /configuration/initializers/core_ext.rb, made the necessary (one-character!) tweak, and overrode the private TextHelper constant with the new Regex. The changed line is in bold:

ActionView::Helpers::TextHelper::AUTO_LINK_RE = %r{
              (                          # leading text
                <\w+.*?>|                # leading HTML tag, or
                [^=!:'"/]|               # leading punctuation, or
                ^                        # beginning of line
              )
              (
                (?:https?://)|           # protocol spec, or
                (?:www\.)                # www.*
              )
              (
                [-\w]+                   # subdomain or domain
                (?:\.[-\w]+)*            # remaining subdomains or domain
                (?::\d+)?                # port
                (?:/(?:[~\w\+@%=\(\)-]|(?:[,.;:'][^\s$]))*)* # path
                (?:\?[\w\+@%&=.;:/-]+)?     # query string
                (?:\#[\w\-]*)?           # trailing anchor
              )
              ([[:punct:]]|<|$|)       # trailing text
             }x

Our test passes, and all is well.

capistrano hanging?

I was a lone developer who was trying to deploy to our staging environment, but my capistrano setup would hang. no one else was having a problem. I traced the problem to a hanging ssh, so I wrote up a simple Net::SSH script to try to ssh to the machines:

Net::SSH.start 'web01-s6.stage', 'build' do |ssh
  puts ssh.exec!('ls')
end

this was working totally fine. I was baffled. I then ran cap under rdebug and still noticed that it was hanging when connecting to all the machines. I then tried duplicating the multithreaded connect:

%w/ daemon01-s6.stage web01-s6.stage /.map do |s|
  Thread.new s do |server|
    connection = Net::SSH.start server, 'build', options
    puts connection.exec!('ls')
  end
end.each { |t| t.join }

hang! so, it was threads + Net::SSH that was causing my problem. I asked around and all the other developers were using ruby 1.8.7p72. I was at 1.8.7p160 (I upgrade a lot). so, I downgraded my ruby (good thing I still had the old one deactivated in macports) and huzzah! no more hangs! I can deploy again: