Integration tests - Githubissues

bollwyvl commented 8 years ago

This adds the test harness from nbpresent for end-to-end integration tests.

It uses nose and casperjs, building on the notebook's jstest.py. I am behind the ball on the 4.0.6 -> 4.1 transition, but i did leave myself some cryptic comments about it, so I should be alright.

My initial plan is to get the tests working locally with local anaconda login privileges. Will continue!

bollwyvl commented 8 years ago

Just updated with better screenshot naming to help track down timing issues (one directory per test).

The more issues I encounter, the more I think I have to mock anaconda.org's requests as fixtures... should be relatively easy, make it testable on anaconda.org, and give us better control over our tests. in return, we'll have to make sure that the output is easily generateable, and get on a refresh schedule to regenerate them... i guess tracking to the anaconda-client version.

bollwyvl commented 8 years ago

Getting somewhere now! Turns out the routes weren't resolving for two reasons:

the extension was not installing correctly in the "pristine" (har) test environment
- for the test I am overloading the server_command, but it could be that is unnecessary
the tornado routes were not properly prefixed on the client
- "/ac-login/" vs Jupyter.notebook.base_url + "/ac-login/"
- no doubt other extensions with a JS component that talks to a custom server route will make this same mistake
- this is important to ensure that extensions can be part of more nuanced deployments (e.g. binder)

at this point, it works against local creds. It may still be of interest to do the appropriate mocking, if only so that the tests work no matter where they are run... once the environment is installed correctly, which is an additional piece of metadata that's missing... once all tests are run inside conda build, this will fall out naturally, and there's no point to doing a requirements-dev.txt, probably.

bollwyvl commented 8 years ago

Hooray, first working linux build, with passing tests, on anaconda cloud!

http://anaconda.org/nbcio/nb_anacondacloud/builds/7/0

:cake:

bollwyvl commented 8 years ago

Ok, this is actually "testing" upload (using a fixture response for both package and release).

The coverage is still reporting pretty low, but it could be that it would be higher if the two test runs were combined, since one handles auth and the other no auth.

I still want to get back my local auth option, as I think it is obviously still valuable to do real integration testing, but having a stable set of fixtures were a key step.

bollwyvl commented 8 years ago

Added the local auth stuff back in, which will be good as a baseline to be used locally, complimented by the mocked integration testing.

By my estimation, unless we want to chase down coverage, this is ready to merge... I would definitely appreciate some feedback from @malev or suitable delegates to see if my approach is crazy.

damianavila commented 8 years ago

Uppsss this was automatically merged when I merge #6... weird...

damianavila commented 8 years ago

please @malev and others... review this one even if it is merged so @bollwyvl can have some feedback on this...

damianavila commented 8 years ago

Uppsss this was automatically merged when I merge #6... weird...

I guess this is somewhat related with the rebase of the PR... but not sure... really weird.

damianavila commented 8 years ago

Yes, the other PR was rebased on this one, this is why it got merged automatically...

damianavila commented 8 years ago

BTW, sorry by the early merge... I should have been figured out you rebased...

anaconda-graveyard / nb_anacondacloud

Integration tests #8