Nightwatch integration!

awatson1978 commented 9 years ago

Hi! Just saw the post by @samcorcos on forms.meteor.com about Gagarin.js testing, and I was excited to see another mocha based test solution in the ecosystem.

So, 6 months ago, I approached Andrei over at Nightwatch, and asked about Mocha integration, and about 2 months ago he was able to finally get it working. I had initially expected that he would encapsulate the Mocha describe/it syntax into Nightwatch; but instead, he managed to adjust Nightwatch so it could be run within Mocha.

At first, this was a bit perplexing, but it seems to have been a smart choice, because I think we might be able to simply require Nightwatch in the Gagarin test script, and get all of Nightwatch's extensive UI testing API.

Anyhow, long story short, Andrei has provided the following code example of running Nightwatch within Mocha.

var nightwatch = require('nightwatch');

describe('Github', function() {
  var client = nightwatch.initClient({
    silent : true
  });

  var browser = client.api();

  this.timeout(99999999);

  before(function() {

    browser.perform(function() {
      console.log('beforeAll')
    });

  });

  beforeEach(function(done) {
    browser.perform(function() {
      console.log('beforeEach')
    });

    client.start(done);
  });

  it('Demo test GitHub', function (done) {
    browser
      .url('https://github.com/nightwatchjs/nightwatch')
      .waitForElementVisible('body', 5000)
      .assert.title('nightwatchjs/nightwatch · GitHub')
      .waitForElementVisible('body', 1000)
      .assert.visible('.container .breadcrumb a span')
      .assert.containsText('.container .breadcrumb a span', 'nightwatch', 'Checking project title is set to nightwatch');

    client.start(done);
  });

  afterEach(function() {
    browser.perform(function() {
      console.log('afterEach')
    });
  });

  after(function(done) {
    browser.end(function() {
      console.log('afterAll')
    });

    client.start(done);
  });

});

Could we do the same thing with Gagarin? Put a var nightwatch = require('nightwatch'); at the top of the test script, and import Nightwatch's higher-level UI testing API?

I think we may have been working towards the same goal, from different directions.

More information here http://nightwatchjs.org/guide#using-mocha

apendua commented 9 years ago

@awatson1978 Not tested yet, but I am pretty much sure you can do it, because Gagarin is just built on top of mocha and there's nothing fancy going on in the background.

So basically, you should be able to use nightwatch for your client-side part of the tests and keep using Gagarin for the server part. At least in theory :)

awatson1978 commented 9 years ago

Right???

I've been reluctant to spin up a vanilla mocha test runner, because it was pretty obvious there was some Meteor-specific functionality missing. But Gagarin seems to have addressed all the missing pieces.... so I'm super intrigued.

I think we may be able to get the following script working, which would be a major milestone for testing.

var nightwatch = require('nightwatch');

describe('Items List Page', function() {

  var client = nightwatch.initClient({
    silent : true
  });

  var browser = client.api();
  this.timeout(99999999);

  before(function () {
    return client.execute(function () {
      Items.insert({_id: 'someFakeId', name: "Foo"});
    });
  });

  it("should be available on server", function () {
    return server.wait(1000, 'until data is propagated to the server', function () {
      return Items.findOne({_id: 'someFakeId'});
    }).then(function (value) {
      expect(value).to.be.ok;
      expect(value._id).to.equal('someFakeId');
      expect(value.name).to.equal('Foo');
    });
  });
  it('should display the item in a list', function (done) {
    browser
      .url('https://localhost:3000')
      .waitForElementVisible('body', 5000)
      .assert.title('Items List Page')
      .waitForElementVisible('body #itemsList', 1000)
      .assert.visible('body #itemsList .listItem:nth-child(1)')
      .assert.containsText('body #itemsList .listItem:nth-child(1) h2', 'Foo');

    client.start(done);
  });
});

apendua commented 9 years ago

Why don't you give it a try and see what happens? :)

The above script is missing server = meteor(); declaration. Also, Gagarin is usually running on some random ports to reduce the risk of collision. The server object will have .getDDPSetup() method that returns a promise which will resolve as soon as the server is started. So you can get the port from there.

awatson1978 commented 9 years ago

It will have to be Thursday, but I'm very excited to give it a try! :)

Will post results in a couple of days. And thank you for reviewing my code and providing some pointers for getting things set up!

awatson1978 commented 8 years ago

Hmmmm... any ideas?

{ [Error: Cannot find module '../build/Release/bson'] code: 'MODULE_NOT_FOUND' }
js-bson: Failed to load c++ bson extension, using pure JS version
 system  searching for test files, using glob pattern `/Users/abigailwatson/Code/ClinicalFramework/hello-healthcare/tests/gagarin/**/*.{js,coffee}`                  
  --- added => /Users/abigailwatson/Code/ClinicalFramework/hello-healthcare/tests/gagarin/foo.js

  added 1 test files ...

  --- building app => /Users/abigailwatson/Code/ClinicalFramework/hello-healthcare ---

 system  using /Users/abigailwatson/Code/ClinicalFramework/hello-healthcare/.meteor/local/build/main.js                                                              

  done building ...

/usr/local/lib/node_modules/gagarin/lib/mocha/gagarin.js:251
      throw err;
      ^

Error: Cannot find module 'nightwatch'
  at Function.Module._resolveFilename (module.js:336:15)
  at Function.Module._load (module.js:286:25)
  at Module.require (module.js:365:17)
  at require (module.js:384:17)
  at Object.<anonymous> (/Users/abigailwatson/Code/ClinicalFramework/hello-healthcare/tests/gagarin/foo.js:1:80)
  at Module._compile (module.js:434:26)
  at Object.Module._extensions..js (module.js:452:10)
  at Module.load (/usr/local/lib/node_modules/gagarin/node_modules/coffee-script/lib/coffee-script/register.js:45:36)
  at Function.Module._load (module.js:310:12)
  at Module.require (module.js:365:17)
  at require (module.js:384:17)
  at /usr/local/lib/node_modules/gagarin/node_modules/mocha/lib/mocha.js:184:27
  at Array.forEach (native)
  at Mocha.loadFiles (/usr/local/lib/node_modules/gagarin/node_modules/mocha/lib/mocha.js:181:14)
  at /usr/local/lib/node_modules/gagarin/lib/mocha/gagarin.js:118:13
  at Array.forEach (native)
  at Gagarin.runAllFrameworks (/usr/local/lib/node_modules/gagarin/lib/mocha/gagarin.js:117:22)
  at /usr/local/lib/node_modules/gagarin/lib/mocha/gagarin.js:227:12
  at invokeCallback (/usr/local/lib/node_modules/gagarin/node_modules/es6-promise/dist/commonjs/promise/promise.js:53:15)
  at publish (/usr/local/lib/node_modules/gagarin/node_modules/es6-promise/dist/commonjs/promise/promise.js:98:5)
  at publishFulfillment (/usr/local/lib/node_modules/gagarin/node_modules/es6-promise/dist/commonjs/promise/promise.js:204:3)
  at flush (/usr/local/lib/node_modules/gagarin/node_modules/es6-promise/dist/commonjs/promise/asap.js:35:5)
  at doNTCallback0 (node.js:407:9)
  at process._tickCallback (node.js:336:13)

apendua commented 8 years ago

I am wondering where exactly gagarin is looking for your modules. It's probably easiest to install them globally. If it does not work for you, you will need to create a custom package.json somewhere and try to follow the same pattern as here:

https://github.com/anticoders/gagarin/blob/develop/test.js

awatson1978 commented 8 years ago

Hmmm...

Nightwatch is installed globally.
I've tried running commands with/without sudo.
Tried cloning the gagarin repository, adding nightwatch to the package.js file, and then adding var nightwatch = require('nightwatch') to various files such as test.js and lib/mocha/gagarin.js.

Same error in each situation. Looks like integration isn't going to be quite as easy as hoped (it rarely is).

So, turning the question around... if Gagarin is simply an extension of Mocha, can we do the following to include Gagarin in Nightwatch?

var gagarin = require('gagarin');

Specifically, I'm looking at the following code: https://github.com/nightwatchjs/nightwatch/blob/master/lib/runner/cli/clirunner.js#L361-L386

     case 'gagarin':
        var Gagarin = require('gagarin');
        var mochaOpts = testRunner.options || {
            ui : 'gagarin'
          };

        var gagarin = new Gagarin(mochaOpts);
        var nightwatch = require('../../index.js');

        Runner.readPaths(source, this.test_settings, function(error, modulePaths) {
          if (error) {
            fn(error, []);
            return;
          }
          modulePaths.forEach(function(file) {
            gagarin.addFile(file);
          });

          gagarin.run(nightwatch, this.test_settings, function(failures) {
            fn(null, {
              failed : failures
            });
          });
        }.bind(this));
        break;

Would those test runner options be correct and sufficient? I get the sense that it might spin up the Mocha instance; but all the rest of what Gagarin does might be be used.

Hmmm....

apendua commented 8 years ago

@awatson1978 The Gagarin class is extension of Mocha so pretty much everything you can do with mocha programatically, you can also do with gagarin. A good example is the test.js file I included in the previous comment. Looking at your code here it should pretty much work, but Gagarin constructor may require some additional options like pathToApp, webdriver (if you want to use browser api), etc. Basically speaking, everything you need to know is here:

https://github.com/anticoders/gagarin/blob/develop/test.js#L27-L40

apendua commented 8 years ago

@awatson1978 Do you think we can consider this issue resolved?

anticoders / gagarin

Nightwatch integration! #141