search

WorldBrain / Memex

Browser extension to curate, annotate, and discuss the most valuable content and ideas on the web. As individuals, teams and communities.
https://worldbrain.io
4.43k stars 339 forks source link

Get full browser support set up in Travis CI to run Selenium tests #1026

Open poltak opened 4 years ago

poltak commented 4 years ago

The base Selenium setup implementation was done in #998

It got to the stage where things work in locally installed Chrome or Firefox browsers, though our Travis CI setup needs more work to be able to run the Selenium tests in those browsers.

It seems fairly easy to the headless browsers running in Travis, however due to technical limitations found in the investigation work for this task the headless browsers cannot install and run web extensions. More details on solutions for giving Travis access to full browsers and explanations why headless browsers won't suffice in this comment: https://github.com/WorldBrain/Memex/pull/998#issuecomment-618196445

ShishKabab commented 4 years ago

Did we ever look into some popular existing extension to see how they run their tests?

blackforestboi commented 4 years ago

Do we need to run the selenium tests on the server necessarily? Wouldn't it be possible to do that locally too? meaning starting the extension, running a command in the console that kicks off the selenium work? Afterall these tests don't need to be done so often?

ShishKabab commented 4 years ago

They need to be few, but ideally ran before every deploy. We also want to minimize manual work, so they don't run the risk of being forgotten.

blackforestboi commented 4 years ago

gotya. just wanna avoid that we try to overengineer it. If it doesn't work on every deploy, its also acceptable to do it before every merge of a PR manually.

I am afraid we will lose time to make the system perfect. If we can get it running with doing it much faster I am fine with that for now. Better than nothing.

poltak commented 4 years ago

Did we ever look into some popular existing extension to see how they run their tests?

No we did not. But good suggestion. Perhaps this is a good first step for when we get back to this work

I am afraid we will lose time to make the system perfect.

If we go with that Sauce Labs option, getting it to work with the existing CI setup looks very minimal: https://docs.travis-ci.com/user/gui-and-headless-browsers/#using-sauce-labs (should have put this doc in the OP - very relevant)

The other option I described in that comment was setting up a virtual display via xvfb in our CI setup which Travis can use to run full Chrome and FF in. This will be a bit more time consuming, figuring out how to set it up right, but no reliance on more third-party services.