brigadecore / brigade

Event-driven scripting for Kubernetes
https://brigade.sh/
Apache License 2.0
2.4k stars 247 forks source link

implement a strategy for testing against a live database #1965

Open krancour opened 2 years ago

krancour commented 2 years ago

This issue is one of several that, collectively, will supersede #1811.

We need:

  1. A new suite of tests that are separate from the unit tests (and the existing integration tests) and can be triggered separately from those other two using a new make target. These tests can assume the existence of a live MongoDB at some location like localhost: 27017. Go build tags can help to facilitate this separation and selective execution.

  2. A new job in the CI/CD process that executes the above (brigade.ts). This shouldn't be too hard because Brigade jobs already support "sidecar" containers, so it's just one job composed of two containers where the "primary" executes the tests and the "sidecar" runs a live MongoDB to satisfy the tests' assumption that the database is available at localhost: 27017.

  3. Individuals tests aren't the focus of this issue so much as developing the overall strategy, but clearly we'll have to build a few tests to prove it all works. (I suggest testing just one of the smaller "verticals" in the API, such as service accounts maybe.) The general approach should be:

    1. Assume the DB server is already running.
    2. Initializing a new DB on the running server for each test case will probably be advisable to avoid test cases impacting one another's results. i.e. Each test case needs to execute in its own little sandbox within the DB server.
    3. Put the DB in known / starting state.
    4. Execute the query or statement.
    5. Make assertions on end state.

I will open separate issues for each of the "verticals" after we have proven we have a satisfactory strategy in place.