start adding integration tests

[dunn]

This automates the first section of manual tests from the testing.md document. I'm using CasperJS since it allows access to environment variables and is a lot easier than Nightmare or the various PhantomJS+mocha integrations.

If you're down with this idea, I can work on the rest of the tests.

[dunn]

These tests can be run by Travis for non-PRs, since the env variables need to be secret.

[jsha]

This is so terrific, thanks! Will take a more detailed look in a bit.

[jsha]

I'm testing this out now, and getting:

Test file: test/integration/signup_logon_test.js
# Sign up and log on
PASS Sign up and log on (5 tests)
FAIL Errors encountered while filling form: form not found
#    type: uncaughtError
#    file: test/integration/signup_logon_test.js:818
#    error: Errors encountered while filling form: form not found
#           fillForm@phantomjs://platform/casper.js:818:62
#           fillNames@phantomjs://platform/casper.js:882:25
#           phantomjs://code/signup_logon_test.js:27:21
#           runStep@phantomjs://platform/casper.js:1615:31
#           checkStep@phantomjs://platform/casper.js:406:28
#    stack: not provided

Do you have any advice on further debugging Casper scripts? I'm quite unfamiliar with the tool. I do have a Block Together instance running correctly on localhost.

[dunn]

Oops, I should've added some examples.

I started the Block Together instance on port 3000, then set the BT_TEST_MAIN_USER and BT_TEST_MAIN_PASS env variables and ran npm test:

BT_TEST_MAIN_USER=testtwitter BT_TEST_MAIN_PASS=somethingsecret npm test

[jsha]

Yep, I got that far before getting that stack trace. I was able to make progress by changing https://api.twitter.com/oauth/authenticate to https://api.twitter.com/oauth/authorize. When I click the Log On link, I get a page with a form pointing to authorize, not authenticate. Do you see the former?

With the fix above I get as far as:

  casper.waitForSelector('.container-fluid', function() {                                                                                                                                             

Which fails with a timeout. Is there any way for me to interrogate the state of the browser at the time of that timeout?

[dunn]

When I click the Log On link, I get a page with a form pointing to authorize, not authenticate. Do you see the former?

I do; that's weird.

Is there any way for me to interrogate the state of the browser at the time of that timeout?

Yeah. Casper is weird in that you have to use this.echo instead of console.log, but you can dump the page HTML with this.echo(this.getHTML()) (http://docs.casperjs.org/en/latest/modules/casper.html#gethtml), or use this.evaluate to fetch bits of the DOM.

[dunn]

I don't think anything in the waitForSelector block will be executed until it's satisfied, so maybe add another block beforehand for debugging:

casper.then(function() {
  this.echo(this.getHTML());
});

casper.waitForSelector('.container-fluid', function () {

[jsha]

Thanks, the echo HTML trick helped a lot. What I'm finding is that, in my setup, clicking "Log On" a second time takes me back to https://api.twitter.com/oauth/authenticate?oauth_token=XYZ, which doesn't automatically redirect back to localhost. I'm guessing this may have something to do with the configured OAuth callback URL being http://localhost:3000/auth/twitter/callback in dev, vs https://blocktogether.org/auth/twitter/callback. Did you test these scripts in dev, or against prod?

[jsha]

Alright, I got the tests to pass locally. Can you cherry-pick in https://github.com/dunn/blocktogether/commit/a1f43c55a3a0653efceaffdb3681a0e810c1cf8d and see if that works in your local development environment? If so we can merge and then work on Travis integration. Thanks again for working on this!

[dunn]

Just a few more tweaks and it works on my end too; I left comments on the commit.

[jsha]

Great, pushed another commit to my dunn-int_tests branch. Try that one, then I'll merge it?

Also it would be helpful if you could let me know whether you're running tests against a local dev instance or the prod instance, to help figure out context on any differences.

[jsha]

Whoops, had missed one of your comments, now fixed with my latest push (2198398).

[dunn]

let me know whether you're running tests against a local dev instance or the prod instance

Oh, sorry. I'm running the tests on a development instance set up like so: https://github.com/jsha/blocktogether#macos

[jsha]

Awesome, thanks. Does the latest revision pass for you?

[dunn]

Oddly, I get one failure now:

Test file: test/integration/signup_logon_test.js
# Sign up and log on
PASS Sign up and log on (6 tests)
PASS log on form is present
PASS new account has default settings
FAIL block_new_accounts was saved
#    type: assertEquals
#    file: test/integration/signup_logon_test.js
#    subject: [false,false,false,true]
#    expected: [true,false,false,true]
PASS after logging out and in, settings are preserved
PASS signing up again overrides old settings
PASS there is a valid show-blocks URL
FAIL 6 tests executed in 6.282s, 5 passed, 1 failed, 0 dubious, 0 skipped.

I'll investigate.

[dunn]

On a different machine, though, it passes. So I must have just misconfigured something.