search

loculus-project / loculus

An open-source software package to power microbial genomic databases
https://loculus.org
GNU Affero General Public License v3.0
37 stars 2 forks source link

Refactor backend tests to allow parallelization #2924

Open corneliusroemer opened 1 month ago

corneliusroemer commented 1 month ago

Would be nice if we could run backend tests in parallel, right now some tests seem to require an empty database which is problematic.

Related to #2710

fengelniederhammer commented 1 month ago

I think the main point that needs to be answered here is whether we want:

  1. 100% reproducability of tests (because they always start with an empty DB)
  2. resilience and independence of the tests at the cost of reproducability (i.e. assuming that the DB is not empty and writing tests such that they still run green)
    • Advantage:
      • They might catch some more real-life bugs, because at runtime the DB also is not empty
      • you can run them in parallel
    • Disadvantage:
      • Tests in CI might fail, but it's not reproducable locally, since it might depend on the state of the DB.
      • Tests are maybe a bit harder to write, because they need to be such that they ignore already existing entries in the DB

I would also favor 2. I'm just saying it needs to be a conscious decision.

corneliusroemer commented 1 month ago

Can't we just have a separate db for each test file, or for each process/thread? I don't understand enough (yet) about our setup, but that way it might be possible to avoid the (low) risk of issues (I've run tests in parallel and didn't see any issues, even without any extra db setup in the past).

fengelniederhammer commented 1 month ago

I've never done test parallelization with gradle and testcontainers. I don't know how it would behave. And I can't judge which would be the easier approach.