/dev/mull

How to use context and describe in Rspec

2012-12-16T00:00:00+00:00

I've never been super-clear on how to use context and describe in Rspec. A colleague recently clarified it really nicely for me:

Context is an English version of what the setup is for that group of specs. The nouns around your specs.

Describe is for the verbs (methods) and/or the nouns (objects) which you are testing.

Checking out Puffing Billy

2012-10-12T00:00:00+00:00

Puffing Billy gives you the ability to stub out remote sites in your request (integration) specs, in the same way that you'd use Webmock or Artifice in your unit/controller specs.

I thought I'd throw a little non-rails spec together to check it out. First, some gems:

gem 'puffing-billy', :require => 'billy'
gem 'rspec'
gem 'capybara'
gem 'selenium-webdriver'

Some config:

require 'rspec'
require 'capybara'
require 'capybara/dsl'
require 'capybara/rspec'
require 'selenium/webdriver'
require 'billy/rspec'

Capybara.javascript_driver = :selenium_billy

A spec:

require 'spec_helper'

describe 'Going out to Facebook, and ting', :js => true, :type => :request do

  it "should not hit Facebook" do
    proxy.stub('http://www.facebook.com/').and_return(:text => 'Foobar')

    visit 'http://www.facebook.com/'
    page.should have_content "Foobar"
  end
end

Simple, and very cool. Should make those request specs run a lot faster. Grab the files here.

A use for cucumber

2012-03-23T00:00:00+00:00

I first came across Cucumber at the 2007 QCon conference in London. Aslak Hellesoy gave a really great talk where he took suggestions for an app, and built it live in a 1-hour session in front of everyone there using Cucumber to drive the features - BDD style.

My immediate reaction can basically be summed up as:

OMG This is awesome. 
We'll be able to really engage the product guy 
in writing stories as feature files! 
It'll be so easy for him! 
Just write in plain English 
and, well, Job Done.

I went back to my team, hugely enthusiastic about the potential of the project.

That lasted about an hour.

Turns out that it isn't plain English. To a developer it looks pretty close, but to a normal human you might as well be asking them to write actual code. They have to deal with indenting, for one thing. Also, a big thing with product guys is that they like to mock up visuals on how the thing should look.

So, we just battled on, writing feature files; getting lost in badly organised step definitions. In the end, we did what many teams have done. We went back to Test::Unit and the Product guy did stuff on a whiteboard.

Ultimately, the feature files were only ever
read by a developer and they served solely to 
add a layer of useless abstraction over 
the actual tests.

Fast forward a few months, and we're trying to use Capybara to do some high level integration tests. The whole Capybara/Selenium stack is pretty good testing non-javascript pages, but in trying to get it to drive our Javascript-heavy app we ended up spending more time handling flakiness in our tests than actually providing value.

Now, I've changed companies and moved onto a much larger engineering team. Large enough to warrant having a dedicated automated testing team in-house. There are a couple of interesting things about this. Firstly, having a dedicated resource to handle high-level integration tests sounds like a very good idea because it frees up developers to focus on development and provides a middle ground between Product and Development to ensure someone is always considering the user flow.

Secondly, and more interestingly in the context of this post: These guys are using Cucumber (and Capybara), but they're using it for themselves. I asked them why they did this: Don't you feel it is just a useless layer of abstraction?

Here's the answer:

It helps us get in the mindset of a user.

Now, that seems to me to be reason enough. :)

The Developer Sales Funnel

2011-10-26T00:00:00+00:00

Yesterday, I attended Mashery's Business API Conference which focussed on the evolving nature of APIs and how businesses use them.

Businesses like Twilio, Datasift and Qwerly use their website purely as a lead generation tool. The API is the revenue generator. Twilio CEO Jeff Lawson made some great points about the connection between developers as consumers of APIs and what developers are looking for in an API.

Awareness of a new API in a business often comes from developers, rather than from the marketing or commercial side of the business. This grass roots traction is an important aspect of an API only business. If developers are becoming customers in that way, there must be a developer sales funnel.

The traditional sales funnel looks like this:

Marketing => Visitor => Signup => Pay

It is a model to track the progression of people engaging with your business from their first contact, to finally becoming a customer. It is a funnel because you have less people engaging with each step. (Utopia would be a sales cylinder.) Once you've got someone to the bottom of the funnel, attention changes to retaining them as a customer. That's the holy grail of the subscription business model - but customer retention is a whole other subject.

A funnel where developers are the customers has the same headline points as the tradition sales funnel, but the details behind each step are both interesting and subtle. We have to take into account here that developers are usually smart people, perhaps with a touch of cynicism, and are highly sensitive to marketing material.

Marketing

A developer will hear about a new API from grass roots sources, rather than from a sales email, phone call or glossy brochure. Typically, that will be from another developer, perhaps at a Hacker News meetup, or from an online source like Stack Overflow or r/programming. Typically, you'll hear about the same new hotness a few times from different sources before it hits enough of a threshold in your brain to prompt you to properly check it out.

Visitor Engagement

Once the developer has gone to your website, they'll want to know two things:

* Can I try it out easily?
* What does your API look like?

A developer doesn't want a sales person to reach out to them to learn more about their API needs and budget. They want to know whether it is a RESTful API, whether it uses http properly* and whether they can sign up and get a token easily - just to try it out. So, a clear signup form is important. Good documentation is the channel the developer requires to learn how to actually use the API. Putting effort into clearly written, up to date documentation is well worth it (and unfortunately something many API providers currently fail on) because it is such a key point of decision making for a developer at the start of their engagement with your business.

Trial

Appealing to developers with modern API interfaces such as JSON over RESTful http calls, rather than hugely verbose SOAP/XML calls will help early adoption. That's because the developer has to write less code to start getting results from the API. Current trends are towards using WebSockets which give you bi-directional persistent streaming - great for receiving messages over a long period of time.

Pay

So, once a developer has tried your API and found it easy to use and useful for their needs - the next stage for them is to start paying for their calls. Perhaps that event happens at the end of a trial period, or after their initial credits have been used. At this point the developer is likely to have to make a case to their business representative about the benefits of the API. A clear pricing model, published openly on your website will aid this part of the sales funnel. Again, requiring a developer to speak to a sales representative presents more of a barrier than an enticement to adoption.

* Footnote: DELETE is not a GET request.

Serving Rails apps with Unicorn and Bluepill. Includes Resque.

2011-07-16T00:00:00+00:00

At Wordtracker we run four Rails apps in production. Two of those use background jobs with a Resque implementation. We've recently switched to serving them using Unicorn, with Nginx as a proxy. We've found the restart times on deployment to be far quicker and more graceful, which gives us confidence to roll out new features to our customers as soon as they become available.

I hope the following configuration snippets might be helpful for anyone wanting to serve their Rails applications using Unicorn.

For the record, we're currently on Rails 3.0.7 with Ruby 1.9.2.

The forking blocks feel like code, rather than configuration, but they are extremely important if you're running with multiple worker processes:

    pid_path = "tmp/pids/unicorn.pid"
    listen "*:8031"
    worker_processes 8
    timeout 30
    preload_app true
    pid pid_path
    stderr_path "log/unicorn-err.log"
    stdout_path "log/unicorn-out.log"

    before_fork do |server, worker|
      ActiveRecord::Base.connection.disconnect!
      old_pid_path = "#{pid_path}.oldbin"
      if File.exists?(old_pid_path) && server.pid != old_pid_path
        begin
          Process.kill("QUIT", File.read(old_pid_path).to_i)
        rescue Errno::ENOENT, Errno::ESRCH
          # someone else did our job for us
        end
      end
    end

    after_fork do |server, worker|
      ActiveRecord::Base.establish_connection
      rails_env = ENV['RAILS_ENV'] || 'production'
      worker.user('app', 'app') if Process.euid == 0 && rails_env == 'production'
    end

We need to monitor our apps together with their Resque background jobs. We're using Bluepill for that:

    app_name = 'my-app'
    worker_queues = ['my-app'] * 4

    rails_env = 'production'
    user = 'app'
    group = 'app'
    rails_root = "/var/apps/my-app/current"
    ENV['PATH'] = "/opt/ruby192/bin:#{ENV['PATH']}"

    Bluepill.application(app_name) do |app|

      app.working_dir = rails_root
      app.uid = user
      app.gid = group

      app.process("unicorn") do |process|
        process.pid_file = "#{rails_root}/tmp/pids/unicorn.pid"
        process.stdout = process.stderr = "#{rails_root}/log/bluepill.log"
        process.environment = { 'RAILS_ENV' => rails_env }

        # Unicorn needs to be invoked using a path which includes 'current', 
        # otherwise it tries to restart with the executable installed in an old release dir (which get cleaned)
        process.start_command = "bundle exec #{rails_root}/vendor/bundle/ruby/1.9.1/bin/unicorn -Dc unicorn.rb"
        process.stop_command = "kill -QUIT {{PID}}"
        process.restart_command = "kill -USR2 {{PID}}"

        process.start_grace_time = 30.seconds
        process.stop_grace_time = 5.seconds
        process.restart_grace_time = 13.seconds
      end

      if defined?(worker_queues)
        worker_queues.each_with_index do |queue_list, i|
          app.process("resque-#{i}") do |process|
            process.group = "resque"

            process.pid_file = "#{rails_root}/tmp/pids/resque-#{i}.pid"
            process.stdout = process.stderr = "#{rails_root}/log/resque-#{i}.log"
            process.environment = {
              'RAILS_ENV' => rails_env, 
              'QUEUE' => queue_list
            }
            process.start_command = "bundle exec rake resque:work"
            process.stop_command = "kill -QUIT {{PID}}"
            process.daemonize = true
          end
        end
      end
    end

Finally, a bit of Nginx config

    root /var/wordtracker/apps/my-app/current/public;
    access_log /var/log/nginx/my-app.access.log main;

    location / {
      try_files $uri @unicorn;
      access_log off;
      expires 30d;
    }
    location @unicorn {
      proxy_pass http://localhost:8031;
      error_log /var/log/nginx/my-app.error.log warn;
      include /etc/nginx/nginx-proxy.conf;
      proxy_set_header X-Forwarded-Proto https;
    }

Teams and the art of leadership

2011-06-26T00:00:00+00:00

I recently finished reading Mike Brearley's "The Art of captaincy" (Pub 1985, ISBN-10: 0752261843). Brearley is a former cricketer who led the England cricket team in 31 of his 39 Test matches, winning 17 and losing only 4. He was described by team mates as "having a degree in people" and now works as a leading psychoanalyst. The Art of captaincy, ostensibly about cricket captaincy, transcends sport to become a treatise on teamwork and leadership. When reading it, I realised that many aspects and issues he deals with in the book apply directly to leading a team in disciplines outside sport. For example, Sam Mendes mentions in the foreword that he consulted it when making "American Beauty".

I found myself making notes as I went through the book, detailing passages which seemed to apply most directly to my work: leading a team of software developers. Restructuring abstract notes made from Brearley's already terse, clinical writing has been a really interesting challenge. I hope that what follows is of interest to software developers and those trying to lead them.

Team sizes

A team should be small enough for communication to be simple, for individuality to be maintained and for each team member to know the others well. It is not accidental that sporting teams range from around seven to 15 members. A true group cannot exist with less than that and more becomes a crowd. Two is not a team. They may combine well together, but too much hangs on the individuals and how they happen to get on. Threes tend to divide into a two and a one; fours into two pairs. Five or six people can make a group, but the absence of one or two individuals (e.g. for illness or holiday) makes its identity precarious.

Balancing inputs

The leader has the benefit of ideas from all sources. If they differ from his, he discovers where the opposition lies, and what form it takes. A leader will be confronted with sometimes intense pressure from the team to fit in with the prevailing or desired emotional mood. These demands may be conscious or unconscious. A leader may be terrorised into an approved set of attitudes. A leader may be pulled into a mood of delinquency or persecution in ways which undermine the performance of the task.

Giving feedback

Different team members need different forms of praise. Praise offered too freely or too unctuously become devalued. A leader should get to know people's strengths and to learn which sorts of questions to ask of which team members. A leader's input, even in an informal setting, may set creative ideas going, or give a team member a sense of his real value.

On team members

Team success is the product of personal successes. A person's efforts should be valued by the rest, particularly when their contribution is unglamorous. Team members learn the habit of thinking for themselves and their self-respect is enhanced when their right to be heard is fully accepted and the space given to them to speak helps them formulate good responses.

A team member feels the benefit of security of being well regarded and confidence to respond to the realistic but challenging standards which are opened up for him. Roles may also restrict and the same role may work well or badly at different times. A team member can fail to value himself enough, leading to diffidence in the team. Individual protagonists take their meaning from, and are strongly influenced by, the group context.

Difficulties within a team

The atmosphere working in a difficult environment promotes denial. Worries are underplayed and a cynical attitude with an emphasis on efficiency predominates. The outcome is often friction, illness among staff and high levels of mistakes. At times of anxiety and stress the same qualities that have been helpful and vital are inevitably exaggerated or distorted.

The pull of the team may be into a position that is the opposite of the emotional orientation of those making the demand. The leader may be lassoed into an answering emotional attitude. The pressure may be put on him to become a tyrannical boss who relieves the others of the need to think for themselves. Or he may be required to become passive and cringing before a powerful bullying process predominant in a group at that particular time.

A way of lessening guilt is to find ways of making others share it or take it on. The outcome of succumbing to such pulls is that the clarity that leaders need to do the job properly has been hijacked. Power now lies with the unconscious group, with the leader only the leader in name. It is hard for someone who is caught up in the turmoil of pulls to see what's going on, much less to be able to understand the forces in play and regain control.

On meetings

Consultation with team members takes different forms and can be used for different ends. It may merely be the means by which the leader gains technical or practical information on which to base his decision. It may also be the occasion for a decision. Or it may be seen, and used, more as an end in itself, a process of sharing and sorting out feelings, antagonistic or otherwise, within the group. We are right to be skeptical of meetings, but a good team meeting can have the output of a renewed commitment to each other and the side.

When things aren't going well

If a team member is making silly mistakes all consultation, delicacy of feeling, weighing up of pros and cons need to give way to orders, bluntness, decisiveness. Discipline, will power, effort and guts hang together when under pressure. When these are alone the hallmarks of a leader too much is lacking; they represent a narrow view of human nature and of the possibilities of motivation; but without them, leadership is toothless.

An insecure team member may find a degree of acceptance by means of assuming the role of court jester, which provided a feeling of security, however precarious. The court jester behaviour can also feed a malicious humour in the rest of the team, who can get on with their own jobs while enjoying vicariously some of the jester's altercations with authority: the group push him further into the part

Team members may attempt to deal with their doubts by giving up; being passive or depressed - thereby placing a demand on the leader to initiate and take responsibility for everything. This may trigger a leader into becoming bossy and demanding, which perversely the team may have provoked in order to mock.

One sullen or indolent team member can stand out like a sort thumb - her private interest may conflict with that of the group. Enable the team to reach creative solutions without denying their anxieties. To bring people together in a common task so that they take pleasure in their joint and individual work.

Finally: On leaders

A team leader is bound to be the recipient of emotional demands and pressures from those she is responsible for. A good leader or manager is interested in what makes people tick, particularly when they seem to be difficult or withdrawn or under-achieving. This kind of interest and attendant attitude, involve the capacity to step back from the hurly-burly of the group. The leader sacrifices the pleasures of being one of the boys, indulging in the dramas of gossip and having a go at the boss.

The leader is influenced by the team character, but set apart from it, responsible to a degree for moulding it. She is of it, but separate from it, required to be part of it, acting within it whilst overseeing the actions of team members.

Leaders cannot avoid discomfort - they are required to hold in balance conflicting aims and values. A leader should maintain a reasonable balance herself, but being balanced does not mean being flat, innocuous or bland. No team leader can bring out the best in all team members. Temperamentally, we all respond better to some than to others. There are those who impose on us pressures that make us particularly touchy, or which arouse our own anxieties unduly.

Leadership calls for complex qualities which are in conflict with each other: Passion and detachment; vision and common-sense. An authoritarian streak and a truly democratic interest in team and points of view. Conviction but also the capacity to not rush in; the ability to tolerate doubt and uncertainty.

The leader has to be able to take in and think about the anxiety of those who work in the team. It is important to convey that their predicament and anxieties are bearable. If this is done, the team is less likely to resort to damaging states of mind such as defeatism, lack of initiative or complacency. Nor will it be so likely to split into antagonistic factions. This function, of containing anxiety and handing it back in a form that can be thought about, is based on that offered both by our first nurse - the mother to her baby, and subsequently by both parents throughout childhood, adolescence into adulthood.

It's hard to get people to pay for software

2011-04-23T00:00:00+00:00

We've got used to free software. Either we expect it to be free, or (as with Facebook and Gmail) we're happy to use our personal data as currency.

There's a ton of interesting thinking already on the web on the psychological shift that's happened in SAAS (Software as a Service) over the past, say, 5 years, but that isn't really my area - I'm much more interested in building software which people are happy to pay for. I thought it would be interesting to go through a very simple model of financing a new software service from conception to 12 months after launch.

Why would you pay for software?

For someone to subscribe to a software product, that software must increase their revenue, productivity or output in some way by a value greater than the cost of the product. That could happen directly: perhaps by giving you some actionable data points which provide an immediate return; or indirectly: for example by making your internal processes more efficient. That assumption alone dictates that paid-for SASS is almost exclusively B2B.

There's tremendous energy amongst software developers right now to create "Product". There are many threads on Hacker News similar to this one. It details a software developer with, I'm sorry to say, massive naivety - develop a software product and then puzzle over why so few people subscribed. Creating value in software is difficult. Beyond the initial concept you need knowledge of how to reach out to your audience and of course the software development skills to build the product. You'll need design skills to make it look good too.

Bootstrapping a product - some assumptions

Let's take a very simple model and work through what's possible. Lets say you're a software developer and you've got a great idea for a software product. To keep things simple, I'll make some assumptions:

You're convinced that it merits a $20/month subscription and that there's a potentially big market.
You think it'll take 6 months to build the app to a point where it has enough features to launch. (The minimum viable product.)
You can't do it all by yourself. You need a couple of collaborators too. That might be another developer and a designer; or another developer and a customer support/marketing person. Still - lets assume a team of 3.
The three of you are going to give up your day-jobs and work on this full time because you all believe in the project and its potential.
You each need an income of, say, £3000/month to live off. Because you're highly skilled software developers and, well, you're worth it.
Let's not include any other costs, such as rent, incoming data or hosting.

I'm sure we can all quibble with those very broad brush assumptions, but I think they're provide us with a good enough baseline to do some projections.

So, you've got to launch day. You've done a fantastic job on getting the word out there, getting people to link to you, sign up to an alpha, a mailing list and generally building up expectation around the general awesomeness of your shiny new product. You've also built the product.

You get 20 paying customers in the first month and you're absolutely stoked. Over the next 12 months the three of you keep working on the product full time: adding more features, responding to customer feedback, fixing bugs, marketing the product, doing PR, speaking at conferences etc. Generally working your asses off.

Washing up the numbers

Let's be really positive about projections at this point and say you're going to have 1000 paying customers just 12 months after launch. That would be a wild success, right? Well here's how the numbers play out:

Oh. 12 months after launch, you're £77,000 in the red. You tipped the £100,000 overdraft point just after launching.

Lets look at the hockey stick view of those numbers:

We're in a fix here, team. That hockey stick isn't the right way around yet.

Where from here?

Apart from the (impossible) task of getting a bank manager to agree a £100,000 overdraft with no collateral, you've got the difficulty of deciding whether to continue with this venture. The flawed assumption that the user count will keep rising linearly could be especially dangerous. If a competitor came along with a bigger market presence that's going to make things very tough. At least if you've got this far you've a fighting chance of increasing your retention rates and gaining new customers.

Of course, it is this predicament that Venture Capitalists try to solve. They throw money at the problem so that you can get to market faster, to learn sooner rather than later whether you're going to be viable in the long term. In backing high numbers of similar companies, they hope to find one hugely successful star, thereby cancelling out the losses from the predominant group of also-rans. In taking VC funding, you sacrifice the lion's share of your business in the hope that the amount you're left with will be worth more in the long term as a result of their investment. That's a whole other business model.

So that's a very quick and simple view of bootstrapping a startup. I deliberately haven't touched on freemium models, conversion/retention rates or launch methods. If you're considering going through this process for your own awesome concept then these are the key parameters:

How many people are involved?
How little are you willing to pay yourselves?
What is your minimum viable product?
How soon can you launch?
How will people get to hear about you?
Why is your app worth paying for, and how much?
How many subscribers will you be able to attract?

That last point is the most important, and that's the one you don't have control over.

Rsift - A Ruby client for Datasift

2011-01-01T00:00:00+00:00

DataSift is a "real time social media filtering engine". They have recently published an API which allows developers to create and manage "streams".

A stream in Datasift context is really a filter over their fire-hose of incoming data from Twitter and other media sources. Another part of their API presents a WebSockets implementation for a client to consume data which matches a stream or filter.

The domain of social media curation is very hot right now, so I was very interested to receive a alpha API key to try it out. I work mainly with Ruby, and there wasn't a client out there yet to "wrap" the Datasift API in Ruby, so I thought that writing that gem would be my first step.

The gem can be downloaded from rubygems and the code is open source, and available at github.

There are three main facets to the API: Streams, Comments and Data. Their API documentation takes us through the semantics of each object.

My first port of call was to create a mechanism to send an http request to their API endpoint. Interestingly, Datasift are calling it a REST API, but actually every message is a GET - even creating an object such as a new Stream. I decided to create a class for each API Object: Stream, Comment, Data; which each have a Model superclass to instantiate the endpoint url, api key and username.

From there, it was a question of understanding the URL design for each object. I chose to create a method, simply called 'do' on each object which sets up the correct URL for each method and calls it.

I've ended up with this pattern:

stream = Rsift::Stream.new(api_url, api_key, username)
response = stream.do("my") # lists my streams

opts = {:stream_id => "1"}
response = @stream.do("get", opts) # gets a stream, by id

The websockets implementation was new to me. I found the em-http-request part of EventMachine on github, which made it incredibly easy to implement a client listener to the Datasift websockets server:

def self.perform(stream_identifier)
  endpoint = "ws://stream.datasift.net:8080/"

  EventMachine.run {
    http = EventMachine::HttpRequest.new(
        "#{endpoint}#{stream_identifier}").get(:timeout => 0)

    http.callback do
      puts "Connected to datasift" 
    end

    http.errback do 
      raise SocketError.new("Datasift threw an error")
    end

    http.disconnect do
       raise SocketDisconnect.new("Datasift disconnected me.")
    end

    http.stream { |msg|
      yield msg
    }
  }
end

This can be called like so:

Rsift::Socket.perform(stream_identifier) do |message|
  puts message
end

I've added in a couple of custom error classes to handle being disconnected by Datasift, or if the socket throws an error.

Please take a look at the test code on github for more usage examples.

By way of a disclaimer, there are a couple of limitations with the code as it stands right now: (and which I hope to find the time to improve!)

All calls require credentials to be passed in
All responses are in JSON

I'd love to get some feedback, please let me know if you're finding it useful.

The Pivot as Prepare, Execute, Listen

2010-11-21T00:00:00+00:00

One of the key principles in lean startup is the Pivot. It has become a bit of a buzzword. The term comes from basketball - keep one foot planted but change direction with the other. The metaphor for business is that you stay grounded in what you've learned whilst testing something new. For example, you might have been offering a monthly subscription product: you could pivot to offering an annual subscription instead, or as well as the monthly.

As I've been reading about Pivots, I realised I had met the pattern before in a quite different context.

Before I was involved in tech, I was a musician. I studied Double Bass at the Royal Academy of Music and played professionally for four years after graduating. Between 1996 and 2000 I was lucky enough to play with the Philharmonia Orchestra, London Symphony Orchestra, Royal Philharmonic Orchestra, the BBC Symphony Orchestra and the Halle Orchestra. I was invited to play at the Salzburg Festival and record for Deutsche Grammophon and Phillips at Abbey Road studios, London.

The principle double bass of the Philharmonia is Neil Tarlton. He taught me about the "Prepare, Execute, Listen" pattern of practise. A musician working on a piece practises by breaking the piece down, phrase by phrase. For each phrase, he'll have a concept of how he wants it to sound: his interpretation of the phrase.

This form of practising is a loop of:

Preparing by refreshing your concept for the phrase in your mind.
Executing it.
Testing the results against the imagining you originally had, by listening.

You may find that you wish to evolve your original idea, perhaps because your interpretation has evolved or to work around a technical difficulty. To me, that's incredibly similar to the concept of pivoting.

As another point of interest, there's something I've noticed which is very different about the tech industry to the music profession. Back in 1997, when I was working with Neil, he published a book of double bass bowing exercises based on the famous Czech violinist Ševčík's exercises for the violin which were first published in 1880. A quick dance around Wikipedia shows that Ševčík was taught by Bennewitz, who was taught by Viotti, who was taught by Pugnani, who was taught by Tartini.

Tartini (1692-1770) is still well known in classical music circles today, partly for being the first person to play one of Stradivari's violins but mainly for writing fiendishly difficult violin music. The most famous piece he wrote was the Devil's trill, which I notice has 4,919 scrobbles at the time of writing.

My point is that there's a direct lineage of pedagogy between Tarlton and Tartini which covers almost 300 years. To a musician that's completely normal. A violinist playing the Devil's trill today has the same challenges as in 1750 or 2350. They may even be using the same instrument to play it. In tech, however, this doesn't really exist. Of course I understand why it doesn't exist, and I know that we have Babbage, Lovelace and Turing; more recently Knuth and Stallman (to name a few) as pedagogues of a sort, but the speed of evolution of technology means that we're doing very different things in our work today, than they were. I find that the tech industry is largely populated by young people who can be very sure of themselves: "Hey, we've been doing this for ten years now!" The vibrancy and energy around the startup movement is infectious and engaging, but there's sometimes a tendency towards very black and white views that 'X' is right and 'Y' is wrong. I think we miss much from not having the pedagogy of an older profession and the respect which is inherent in it.

Running background jobs with Resque

2010-11-03T00:00:00+00:00

Our users at Wordtracker search for keywords and linking domains many thousands of times each day. We run the majority of these searches as "background jobs" to maintain a good user experience.

We'd developed an in-house system written in Ruby and Sinatra for running these jobs, but the robustness and error reporting wasn't as good as we wanted. I decided to take a look at the Redis backed Resque (pron. res-queue |ˈreskyoō|) to see whether it would work well for us.

The first thing to understand when looking at a background job runner like Resque, is the aspects of the process you can hand off and the aspects which you still have to take responsibility for within your application.

To work this out, here's the process flow from a Wordtracker search:

A user hits a search button somewhere on one of our tools.
We check that no job is currently running, for that search type, for that user.
We start the job. (It is at this point that we use Resque to fork into the background process).
The job runs.
The user sees the results.
We clean up the job and archive it.

So, the application has to keep track of all jobs - particularly the state a job is in. Resque is only responsible for the actual running of a job.

Lets break down the resque part of the process:

After doing the "check" mentioned above, we drop into the run method on the Job model, which looks like this:

def self.run(user_id, name, args = {})
  job = Job.create!(:user_id => user_id, :name => name, :state => "new")
  Resque.enqueue(Jobs.const_get(name), user_id, args)
  job
end

The enqueue method is where we fork to Resque. The name of the job that's passed into that method matches one of the class names which are Resque jobs. We've put all our Resque job classes in a file lib/jobs.rb. For example, we have a job called "MetricsList", which will be passed in as a name, and match a class like this:

class MetricsUnsavedList < ResqueJob

The other parameters are the id of the user who kicked the job off, and a hash of job-specific options. It is a common idiom in Ruby to present arguments as a hash when the contents of the hash might vary and to initialise an empty hash to make the parameter optional.

A Resque class must respond to the perform method and set the queue name. Because there are many different types of search a user can do using Wordtracker, we decided to create a ResqueJob class to DRY up common aspects of job handling - those that happen before and after the actual business of running the job. For this transactional pattern, yielding inside a block is helpful, we've wrapped a while_updating_job_status method inside each perform call to handle these functions:

name the job
set it to a running state
if we're still runnning, set to complete
handle any errors

Here's what that looks like:

def self.while_updating_job_status(user_id)
  job_name = self.name.split("::")[-1]
  @job = Job.find_by_user_id_and_name(user_id, job_name)
  @job.update_attribute(:state, "running") if @job.state == "new"
  begin
    yield
    if @job.state == "running"
      @job.update_attribute(:state, "complete")
    end
  rescue Exception => e
    handle_job_error(e.message)
    raise
  end
end

The reason for the funky self.name.split("::")[-1] is because we needed the namespace of the classname to be dropped - can anyone think of a better way of doing that?

Then, the perform call inside the MetricsList class looks like this:

def self.perform(user_id, args)
  while_updating_job_status(user_id) do
    # actual search code goes here
  end
end

The reason for the generic exception handling is that the Resque jobs (obviously) happen inside a different process, so errors won't be passed back directly to our main application which started the job. To get around this, we chose to add a message attribute on the Job model and set the state of the job to error. handle_job_error is also called at any point in the main body of the perform method where we think there's a chance an error might be thrown and we want to handle it in a specific way.

We're very happy with our Resque implementation. Since going live a couple of months ago, it has run almost a million searches with a failure rate of less than 1%.