mojotech / pioneer

Integration Testing
MIT License
527 stars 35 forks source link

Should explain better how it actually works #300

Open bitplanets opened 9 years ago

bitplanets commented 9 years ago

The lacking of callback as opposed to the normal cucumber-js is kind of confusing. How one task "knows" when ended or not without a callback? This was the thing that confused me the most.

I'm trying cucumber-js, while is harder to setup (is not out of the box like pioneer) the errors are more descriptive and steps have callbacks.

IMO this lib might do too much magic by removing callback, but a better doc might change my mind.

tomhicks-bsf commented 9 years ago

You'll want to brush up on JavaScript promises as that's pretty much how the whole UI testing lark hangs together. It all starts with webdriver and their choice to use promises, which is a good choice, IMO.

samccone commented 9 years ago

:+1: this needs more docs for sure open to any PRs

bitplanets commented 9 years ago

But I never complete the promises with this lib. They auto magically complete themselves.

tomhicks-bsf commented 9 years ago

Not quite sure what you mean by that. You return a promise from your step definition, and when that promise resolves/rejects, that's the end of the step. Unless we're talking about different things....

bitplanets commented 9 years ago

My features looked like this:

    this.Given(/^I drop the database$/,function(){
        var databaseName    = 'test';
        var mongoose        = require('mongoose');
        var mongooseConnect = require('mongoose-connect');
        var connectData     = mongooseConnect.connect(databaseName, [], mongoose);
        var dropped         = false;
        connectData.connection.on('connected', function(){
            connectData.connection.db.dropDatabase(function(err, _dropped){
                (err === null).should.be.true;
                _dropped.should.be.true;
                dropped = true;
            });
        });
        this.driver.wait(function(){
            return dropped;
        }, 10 * 1000).then(function(){
            console.log('db dropped.');
        })
    });

    this.Given(/^I open the game$/,function(){
        this.driver.get('https://localhost/')
    });

    this.Given(/^I wait for (\d+) seconds$/,function(n){
        var timerDone = false;
        setTimeout(function(){
            timerDone = true;
        }, n * 1000)
        this.driver.wait(function(){
            return timerDone;
        }, Infinity).then(function(){
            console.log('waited done.');
        })
        // this.driver.wait(function(){}, 5000)
    });

    this.When(/^I should see the loading screen$/, function(){
        this.Widget.isVisible({
            selector: '#preloader'
        }).then(function(value){
            value.should.be.true;
        })
    });

This actually works. I don't know how and I'm not returning any promise. I didn't noticed that was necessary to return the promise (as here https://github.com/mojotech/pioneer/blob/master/test/integration/steps/steps.coffee) maybe because are written in coffee.

Now I understand how it works if you return. But in my case how did that worked?

tomhicks-bsf commented 9 years ago

Yeah so coffeescript implicitly returns the last statement in a function.

Your code might work but it's not correct. I suspect you'll be getting things happening out of order because you're not letting the steps resolve correctly. Have you tried changing your #preloader selector to something you know is not in the document to ensure you're not just getting false positives?

You're doing your assertion in your When which means it'll probably just be skipped over and the promise might well resolve after everything has finished. Also, you really shouldn't need any waits in your tests - WebDriver does a very good job of waiting until elements appear before continuing, so you really shouldn't be relying on them.

Try something more like this:

    this.Given(/^I open the game$/,function(){
        return this.driver.get('https://localhost/');
    });

    this.Then(/^I should see the loading screen$/, function(){
        return this.Widget.isVisible({
            selector: '#preloader'
        }).should.eventually.be.true;
    });

Look here: https://github.com/domenic/chai-as-promised/ for information on asserting on promise resolutions.

bitplanets commented 9 years ago

Well they worked in the same order, don't know why. But I still don't know how they worked

Have you tried changing your #preloader selector to something you know is not in the document to ensure you're not just getting false positives? Yes.

So how would you do the wait in my previous example without wait? I'm not waiting for dom elements but specific logic.

should.eventually.be.true Now I understand the eventually.

tomhicks-bsf commented 9 years ago

Instead of waiting for your database to drop like that you could instead return a promise using a library such as Bluebird or Q to help you convert the node style callbacks into promises. It's not much different, except in semantics: you're currently asking the webdriver to wait while the database drops, whereas I'm suggesting you ask your test runner to wait while the database drops.

And in general, you shouldn't have or need a 'wait for N seconds' step, although seeing as you're testing a game I can see that that might be more necessary than testing a traditional webapp.

bitplanets commented 9 years ago

Good suggestion, thanks.