search

conda-incubator / conda-store

Data science environments, for collaboration. ✨
https://conda.store
BSD 3-Clause "New" or "Revised" License
142 stars 46 forks source link

META - testing improvements #662

Closed dcmcand closed 4 months ago

dcmcand commented 9 months ago

Context

This is a meta issue to track improvements that need to be made to conda-store's testing.

Value and/or benefit

Our test suite should provide the following:

  1. Rapid feedback to developers - we want failures to occur as early as possible in the pipeline so developers can get feedback on their changes rapidly.
  2. Relevance - We should test things that we control. In practice this means not testing other people's code. We should be using pinned versions of dependencies for integration tests, mocked responses for end to end tests, and only testing our own code for unit tests.
  3. Consistency - We need to ensure that all of our tests are consistent and deterministic. Flaky tests are annoying, slow down feedback loops, and undermine confidence in the system.
  4. Confidence - By the end of our test pipeline we should have high confidence that what we release will not break things for users and that our code works as we intend. This includes both for functional requirements, and nonfunctional requirements such as security and accessibility.
  5. Comprehensive coverage - all important user journeys and code paths should be comprehensively tested.

Anything else?

No response

### Tasks
- [ ] https://github.com/conda-incubator/conda-store/issues/619
- [ ] https://github.com/conda-incubator/conda-store/issues/668
- [ ] https://github.com/conda-incubator/conda-store/issues/669
- [ ] https://github.com/conda-incubator/conda-store/issues/670
- [ ] https://github.com/conda-incubator/conda-store/issues/671
- [ ] https://github.com/conda-incubator/conda-store/issues/672
- [ ] https://github.com/conda-incubator/conda-store/issues/673
- [ ] https://github.com/conda-incubator/conda-store/issues/674