abba

abba is a simple a/b testing framework for JavaScript and Node.js.

It's a node.js clone with small improvement over @maccman's Abba.

Features

Simple JavaScript API
Multi variant support
Filter results by date and browser

Requirements

Node.js 6+
MongoDB

Getting started

# Clone the project
git clone https://github.com/7anshuai/abba
cd abba

# Install dependencies
npm install

# or if you're using Yarn
yarn

Then you can begin development:

# yarn
yarn run dev

# npm
npm run dev

This will launch a nodemon process for automatic server restarts when your code changes.

Testing

Testing is powered by Mocha. This project also uses supertest for demonstrating a simple routing smoke test suite.

Start the test runner with:

# yarn
yarn test

# npm
npm test

You can also generate coverage with:

# yarn
yarn run cover

# npm
npm run cover

Linting

Linting is set up using ESLint. It uses ESLint's default eslint:recommended rules.

Begin linting in watch mode with:

# yarn
yarn run lint

# npm
npm run lint

Environmental variables in development

The project uses dotenv for setting environmental variables during development. Simply copy .env.example, rename it to .env and add your env vars as you see fit.

It is strongly recommended never to check in your .env file to version control. It should only include environment-specific values such as database passwords or API keys used in development. Your production env variables should be different and be set differently depending on your hosting solution. dotenv is only for development.

Using docker in development

You will need docker and docker-compose installed to build the application.

After installing docker, start the application with the following commands:

# To build the project for the first time or when you add dependencies
$ docker-compose build web

# To start the application (or to restart after making changes to the source code)
$ docker-compose up web

To view the app, find your docker ip address + port 8080 ( this will typically be http://192.168.99.100:8080/ ).

Deployment

Deployment is specific to hosting platform/provider but generally:

# yarn
yarn run build

# npm
npm run build

will compile your src into /dist, and

# yarn
yarn start

# npm
npm start

will run build (via the prestart hook) and start the compiled application from the /dist folder.

The last command is generally what most hosting providers use to start your application when deployed, so it should take care of everything.

A/B Testing API

First include abba.js using a script tag. The host of this url will need to point to wherever you deployed the app.

<script src="https://github.com/7anshuai/abba/raw/master//localhost:8080/scripts/abba.js"></script>

Then call Abba(), passing in a test name and set up the control test and variants.

<script>
  Abba('test name')
    .control('test a', function(){ /* ... */ })
    .variant('test b', function(){ /* ... */ })
    .start();
</script>

The control is whatever you're testing against, and is usually the original page. You should only have one control (and the callback can be omitted).

The variants are the variations on the control that you hope will improve conversion. You can specify multiple variants. They require a variant name, and a callback function.

When you call start() Abba will randomly execute the control or variants' callbacks, and record the results server side.

Once the user has successfully completed the experiment, say paid and navigated to a receipt page, you need to complete the test. You can do this by invoking complete().

<script>
  // On successful conversion
  Abba('test name').complete();
</script>

You can find an example under ./public/test.

options

Persisting results

If set the persist option to true, then the experiment won't be reset once it has completed. In other words, that visitor will always see that particular variant, and no more results will be recorded for that visitor.

<script>
  Abba('Pricing', {persist: true}).complete();
</script>

Weighting

You can set a variant weight, so some variants are used more than others:

Abba('My Checkout')
  .control('Control', {weight: 20})
  .variant('Variant 1', {weight: 3}, function(){
    $('#test').text('Variant 1 was chosen!');
  })
  .variant('Variant 2', {weight: 3}, function(){
    $('#test').text('Variant 2 was chosen!');
  })
  .start();

In the case above, the Control will be invoked 20 times more often than the other variants.

Flow control

You can continue a previously started test using continue().

Abba('My Checkout')
  .control()
  .variant('Variant 1', function(){
    $('#test').text('Variant 1 was chosen!');
  })
  .variant('Variant 2', function(){
    $('#test').text('Variant 2 was chosen!');
  })
  .continue();

Nothing will be recorded if you call continue() instead of start(). If a variant hasn't been chosen previously, nothing will be executed.

You can reset tests using reset().

Abba('My Checkout').reset();

Lastly, you can calculate the test that you want to run server side, and just tell the JavaScript library which flow was chosen.

Abba('My Checkout').start('Variant A');

Links

If you're triggering the completion of a test on a link click or a form submit, then things get a bit more complicated.

You need to ensure that tracking request doesn't get lost (which can happen in some browsers if you request an image at the same time as navigating). If the link is navigating to an external page which you don't control, then you have no choice but to cancel the link's default event, wait a few milliseconds, then navigate manually:

<script>
  $('body').on('click', 'a.external', function(e){
    // Prevent navigation
    e.preventDefault();
    var href = $(this).attr('href');

    Abba('My Links').complete();

    setTimeout(function(){
      window.location = href;
    }, 400);
  });
</script>

That's far from ideal though, and it's much better to place the tracking code on the page you're going to navigate to. If you have control over the page, then add the following code that checks the URL's hash.

<script>
  if (window.location.hash.indexOf('_abbaTestComplete') != -1) {
    Abba('My Links').complete();
  }
</script>

Then add the hash to the link's URL:

<a href="https://github.com/7anshuai/abba/blob/master/blog#_abbaTestComplete">

License

MIT License. See the LICENSE file.

7anshuai / abba

readme