Bunyip: client-side unit testing made easy

Let’s face it doing thorough client-side unit testing fills me with rage, throw mobile browsers into the mix and I want to start flipping tables. There are tools out there to somewhat tackle this issue but they either require painful setups or want you to rewrite your unit tests using their framework. What if I told you there’s a tool that is easy to get up and running, doesn’t require you to rewrite your tests and allows you do it all from the command line in desktop and mobile browsers. Watch a video showing it working.

The codestream video can take a while to load sometimes, be patient.

If you want to get stuck in go check out Bunyip on github and see the readme for instructions on getting up and running. It’s really easy I promise.

Bunyip

At Web Directions Code conference in Melbourne this past May I demonstrated this tool on stage in my talk “Debugging secrets of a lazy developer”. I started thinking about a tool like this when I joined the Modernizr team and how annoying, but important, it was to utilise the powerful test suite we have. Opening all the browsers and visually checking the test suite results is time consuming, thankfully a series of open source tools and a powerful API made this idea become a reality.

On the backs of great projects

The first tool that got Bunyip into a usable state is Yeti an incredibly powerful tool from Yahoo that allows to push out a html file to any connected browsers from the command line. It launches a little server on your machine that browsers on the same network can connect to. The downside initially was that it required you to use the YUI testing framework, but this tool was way too awesome to ignore it on that factor. So I dug in and discovered how it was sending back the results to the cli and wrote some adaptors for QUnit and jasmine to work with Yeti. If you use another framework feel free to do a pull request!

In the Yeti docs they suggest enabling mobile support by using a service called localtunnel which allows you to create an SSH tunnel into your local server so you can access it from outside your local network. I found localtunnel to be slow and sporadic with Yeti especially when the test suite was quite large. I settled on another service called pagekite which gives you 2.5GB of data for free and the ability to register a subdomain after that it’s pretty cheap. You can also check out showoff.io.

Next is a great service called BrowserStack. BrowserStack lets you test web sites in a whole bunch of browsers but more importantly they offer an API so you can spin up a browser and pass it a url. Exactly what I needed to make bunyip amazing. When I started building this tool the API only had support for desktop browsers, just before I did my talk they rolled out v2 which included mac and mobile browsers! Since I’m building this as a node module I’m using the very awesome node-browserstack module by Scott Gonzales to make using the API very easy.

How it works

As mentioned previously this is reliant on a lot of great OSS. To get into the deeper level on how this works you can comb through the code on github or read on for a high level look.

Local server

When you run this command bunyip -f test/index.html -b "firefox:win/12.0|ie:win/8.0|safari:mac/5.1|iPhone 4S:ios/5.0" in bunyip the first thing it does is hand off the -f test/index.html call to Yeti to spin up a local server that will have an intermediate page that will accessible via localhost:9000. Yeti uses socket.io to notify the cli when a slave connects and vice versa to control when a test should begin.

SSH tunnel and BrowserStack

Once Yeti is up and running bunyip will then grab the tunnel service config values from the file you would of setup. In my case it will run the pagekite.py script passing in my port 9000 and my subdomain I want it accessible via bunyip.pagekite.me.

We now have our tunnel up and running. Next we can parse the -b "firefox:win/12.0|ie:win/8.0|safari:mac/5.1|iPhone 4S:ios/5.0" flag into a usable JSON string we can send off to the BrowserStack API. It looks like this:

[
    {
        "browser": "ie",
        "os": "win",
        "version": "8.0",
        "url": "http://bunyip.pagekite.me/",
        "timeout": 480
    }
]

We authenticate with the API by passing in our credentials from the config.js file and instructing it to load up the workers we specified, in our case Firefox, IE, Safari and an iPhone 4S. When the Yeti waiting page loads it will emit an event that will trigger the cli to display a message letting us know that the browser/device has connected and is waiting for our test suite.

Pushing out the test suite

When all devices have connected we push enter to kick off running it through our connected browsers, Yeti will serve the page specified in our -f flag by emitting an event to all connected browser, using socket.io, telling them to redirect to our test suite. On page load Yeti injects a script called inject.js this is for any test suites written in YUI test if you’re not using YUI test then you would have included one of my yeti adaptors for either QUnit or jasmine this captures the test results as JSON in the format Yeti expects and emits an event back to the global $yetify object which sends off the results to the cli to handle what to display back to the user.

You can help make this awesome

Find any issues or want a new feature or think something can be improved. Then please by all means contribute in any way you can. I want this tool to be a community effort.

Other tools

Of course bunyip isn’t the first tool out there to try and do something like this so I’m going to list of some other tools that might be of interest.

What’s next?

Getting this battle tested inside some more real projects, especially people with jasmine test suites (I haven’t done a lot of testing with the adaptor I wrote for Yeti when using jasmine test suites).

Reid Burke (Yeti dude) is keen to get the Yeti adaptors into Yeti core so it’ll work with other test frameworks straight out of the box.

I need help making the config.js process less painful see issue #1.

See roadmap / wanted features wiki for what I want to add.

Client-side unit testing shouldn’t be hard and it shouldn’t get in your way. Hopefully bunyip can alleviate the pain and get people writing unit tests for the first time or more often.

Short URL: http://cssn.in/ja/041

Post filed under: javascript.

Skip to comment form.

  1. Davit says:

    Takes quite some time for agents to connect. Is it possible to reuse already captured browsers for subsequent test runs?

    • Ryan Seddon says:

      @Davit

      Yeah it really depends on the browsers to how quick the VMs can spin up and connect, iOS devices are the quickest. However the BrowserStack API is still in beta and will get faster.

      Recapturing the browsers for subsequent tests is something I’m working on for another release. So you’d spin up the browsers you need with a long timeout value and then you can push multiple test runs to it without relaunching instances.