Delivering Features vs Testing them (and Jasmine Async)

Michael was a little later to our pairing session yesterday and so I spent a little time scratching some itches about the way in which our AgileBot pings the Gitter chat for the “Agile Development using Ruby on Rails” MOOC. I’d previously been frustrated in my attempts to add hyperlinking to the Slack messages since the ‘request’ library seems to escape the Slack hyperlink syntax, and it wasn’t immediately clear how to address that. Any changes to the AgileBot might easily blow up since it’s a legacy app with no tests. Michael and I had been trying to rectify that this week, but we were moving slowly due to lack of familiarity with the CoffeeScript AgileBot is written in; so I took the opportunity of some solo time to have a crack at reformatting the Gitter messages, which looked like this:



No one had complained about them explicitly, but my intuition was that the long hangout link looked messy, and might be intimidating. Of course this is dangerous territory for Agile development. A common part of the Agile approach is to ensure that you are responding to customer needs, rather than incorporating necessary features on a whim. However there’s a big difference between building a system for a paying customer compared to edging a volunteer community towards critical mass. The paying customer gives you a concrete person to satisfy. In a community you’ll get suggestions, but they may not be practical and as the saying goes, they might just tell you they want a faster horse.

Anyhow, the AgileVentures system has lots of little rough edges, itches I want to scratch, and scratching them makes me feel good. Remembering that Gitter supports the full markdown syntax (unlike Slack) I thought there might be a quick win and with a live chat I could easily get user feedback like so:



which got some immediate feedback



and allowed us to evolve the formatting a little further



The change involved a tiny reformatting to a string in the AgileBot, specifically:

send_gitter_message_avoid_repeats room, "[#{req.body.title} with #{user.name}](#{req.body.link}) is starting NOW!"

I pushed it to the staging server, tested it there, saw that it worked on our agile-bot test channel, and then pushed it live. Within vary short order we had the re-formatted messages coming into the channel:



I hadn’t been able to remove the time element, which is an artefact not of the AgileBot, but of the main site. I got a quick pull request in to fix that, but that’s take a little longer to deploy.

Now maybe changing these Gitter links won’t make much difference to the community in the long run. At the very least they made me feel good and motivated on a Thursday. I hope the users in the chat get a positive feeling and are maybe inspired to get more involved in the community, and my biggest hope is that as we re-run “Agile Development using Ruby on Rails” that completely new users will have slightly less friction as they consider joining a pairing session.

I’m particularly glad that I spent the time solo-ing on the above, because the rest of the afternoon pairing with Michael was somewhat frustrating in that we were repeatedly stuck on getting a basic test of the 3rd party HTTP connection for the AgileBot. We did make progress over the course of the session, but nothing that any end user would see soon. The AgileBot hits 3rd party services Gitter and Slack to get its job done. If we are going to do proper integration tests these services need to be stubbed. We trying the node equivalent of VCR, Sepia by the folks at LinkedIn, which will record and save 3rd party HTTP interactions and allow them to be played back, effectively sandboxing an app. We got sepia working, however in playback mode it highlighted cache misses (unexpected network connections) by creating a file rather than throwing an error that could be caught by JasmineNode.

We set that aside and tried Nock, one of many node world equivalents of WebMock that allows precision stubbing of network connections. Personally I prefer approaches like VCR and Sepia that allow blanket recording of interactions, in contrast to WebMock and Nock which require you to write out individual network stubs by hand. We had nock working, but ran up against node async issues. The JasmineNode tests were not waiting for the HTTP connections to complete, and we were tying ourselves in knots trying to get the JasmineNode async to work in CoffeeScript.

We untied ourselves by dropping back to ground truth, by first getting the example Jasmine Async test working in pure JavaScript:

describe("Asynchronous specs", function() {
  var value;
  beforeEach(function(done) {
    setTimeout(function() {
      value = 0;
      done();
    }, 1);
  });

  it("should support async execution of test preparation and expectations", function(done) {
    value++;
    expect(value).toBeGreaterThan(0);
    done();
  });
});  

That working we converted to CoffeeScript and confirmed that that worked

describe 'Asynchronous specs', ->
  value = undefined
  beforeEach (done) ->
    setTimeout (->
      value = 0
      done()
    ), 1
  it 'should support async execution of test preparation and expectations', (done) ->
    value++
    expect(value).toBeGreaterThan 0
    done()

then carefully inserted the elements from our AgileBot HTTP testing setup:

nock = require('nock');

slack = nock('https://slack.com')
                .post('/api/chat.postMessage')
                .reply(200, {
                  ok: false,
                  error: 'not_authed'
                 });

avHangoutsNotifications = require('../scripts/av-hangouts-notifications.coffee')

describe 'AV Hangout Notifications', ->
  beforeEach ->
    routes_functions = {}
    avHangoutsNotifications({router: { post: (s,f) -> routes_functions[s] = f } })
    @routes_functions = routes_functions

  describe 'hangouts-video-notify', ->
    beforeEach (done) ->
      res = {}
      res.writeHead = -> {}
      res.end = -> {} 
      req = { body: { host_name: 'jon', host_avatar: 'jon.jpg', type: 'Scrum' } }
      req.post = -> {} 
      @routes_functions['/hubot/hangouts-video-notify'](req,res)
      setTimeout (->
        done()
      ), 3000

    it 'should support async execution of test preparation and expectations', (done) ->
      expect(slack.isDone()).toBe(true)
      done()

and this would just crash the whole thing. So at least we had identified that the problem was with our code, not how we happened to be implementing Async testing in CoffeeScript in JasmineNode. It was precisely this line @routes_functions['/hubot/hangouts-video-notify'](req,res) that was causing the crash. The one that started the network connection. Having not yet set up for an interactive debugger it was console.log statements all the way down to discover that it was the error reporting roller component under our hood that was actually breaking everything. Clearly that needed to be disabled for testing purposes. That was achieved like so:

rollbar.init(process.env.ROLLBAR_ACCESS_TOKEN, {enabled: false})

and suddenly all the tests went green. It was a frustrating process, but it highlights the problem solving approach of breaking out the different elements of the system you are testing in order to isolate where the problem actually exists. We’ve achieved a much better understanding of the legacy app as a result of all this. I’m just glad for motivations sake that I kicked out a minor improvement direct to the users before we spent the afternoon wrestling with testing frameworks :-)