qutebrowser / qutebrowser

A keyboard-driven, vim-like browser based on Python and Qt.
https://www.qutebrowser.org/
GNU General Public License v3.0
9.76k stars 1.01k forks source link

Rewrite some end2end/BDD tests as unit tests #3319

Open The-Compiler opened 7 years ago

The-Compiler commented 7 years ago

I've talked about this for a while, but never opened an issue for it.

In theory, I think BDD tests (the .feature files in Gherkin) are a good idea, and I've heard of many contributors which find them easy to write.

However, in practice, they're slow and unstable - to the point that some environments are so flaky on the CI that I can't even manage to fix all the flakiness in time.

Over time, I'd like to reduce the count of end2end tests, and have more smaller unit-tests instead. I've done so for the config commands while refactoring that, and I think it's turned out nice.

To get them closer to what an user actually does, it'd even be possible to have a qute fixture or so, so you can do things like qute.run(':set foo bar').

There's probably some stuff that needs to be done before this:

And some refactorings which can probably nicely be combined because they make things more testable:

toofar commented 9 months ago

To get them closer to what an user actually does, it'd even be possible to have a qute fixture or so, so you can do things like qute.run(':set foo bar').

Where we still wanted to do full tests with a real browser we could even keep the .feature files and run the tests in-process instead of in a separate process. Assuming that some of the current issues are around connecting them via parsing logs and dumping state. Eg if we could use signals to implement the bdd steps instead maybe it would be more robust.