search

AllSpiceIO / py-allspice

An AllSpice-API wrapper in Python
MIT License
0 stars 0 forks source link

Test Suite Overhaul #29

Open shrik450 opened 1 year ago

shrik450 commented 1 year ago

Currently, the pytest test suite suffers from the following problems:

  1. It is tightly bound to an AllSpice Hub instance running on localhost with specific credentials.
  2. The tests are order dependent, and you cannot run individual tests and expect them to pass. (Tracked in #49)
  3. No cleanup is performed, so there is random garbage left in the AllSpice Hub instance after the test suite is complete.

Of these, 2 is the highest priority, as being unable to run a single test means you have to run the entire test suite every time.

shrik450 commented 4 months ago

With the amount of integration tests we are adding for the utils, this is becoming even more important, as each tests adds ~30s to the full suite. Ideally we can use a strategy similar to lightbringer's (i.e. vcr) to make these tests deterministic and idempotent.

shrik450 commented 4 months ago

(We can also replace the manual serialization into csv/json with snapshot testing)