Integration tests - Githubissues

zotero / zotero-connectors

Chrome, Firefox, Edge, and Safari extensions for Zotero

https://www.zotero.org/download/connectors

Other

529 stars 124 forks source link

Integration tests #135

Open adomasven opened 7 years ago

adomasven commented 7 years ago

This is becoming pretty critical now. We can get away without unit tests, but we must ensure basic functionality doesn't accidentally break.

Platforms:

[x] Firefox (ESR)
[x] Firefox (Latest)
[x] Chrome (Latest)
[ ] Safari (Latest)

Having at least Firefox ESR would be a good start.

Functionality:

[x] Debug output submission
[x] Report ID submission
[x] Zotero availability check
[x] Translation to Zotero
[x] Translation to Server
[x] Background/inject message passing
[ ] Prompts
[ ] Notifications
[ ] Proxy detection
[ ] Style installation
[ ] Data-files import

adomasven commented 7 years ago

So before I begin, this is the plan:

Use the node.js Selenium Webdriver to drive browsers
Use the Mocha-Chai testing framework (currently used by the Zotero client unit-test suite)
Use Express.js to mock responses from the client, zotero.org and translation targets.

I'll start with Firefox ESR. Any objections/suggestions @dstillman @tnajdek @inukshuk

adomasven commented 7 years ago

The plan has changed to containing the tests within the extension with the debugging build. The webdriver spec and implementations are not quite there yet. We might still use Selenium to actually launch the browsers with extensions preloaded, but that's not strictly necessary. Express.js had been ditched too as we can stub responses with sinon.js since the tests run inside the blackbox.

adomasven commented 7 years ago

Safari automation is stuck upstream. High Sierra (release in 2 weeks) required to launch a session with automation enabled and then it will probably take a while for travis-ci to support it.

Either way, I'll work on tests and we can at least run them manually before releases, which will still catch bugs.

adomasven commented 6 years ago

Seems like we might be able to set up travis automation now, see upstream link above.

adomasven commented 2 weeks ago

We should consider rewriting these since they rely on eval that is no longer viable with MV3, but there have been improvements to headless browsers and selenium since writing this that would allow to automate the browser and get its contents directly in nodejs.