jozu-ai / kitops

Tools for easing the handoff between AI/ML and App/SRE teams.
https://KitOps.ml
Apache License 2.0
266 stars 26 forks source link

Add functional testing for CLI operations #236

Closed amisevsk closed 2 months ago

amisevsk commented 2 months ago

Description

Add functional tests for the CLI and fix minor bugs writing the tests revealed:

The way the tests work is by

  1. Setting up a temporary directory
  2. Configuring Kit to do everything within that directory (i.e. storage is within the directory, etc.)
  3. Running Kit commands directly within that directory and verifying behaviour

For testing pack/unpack and ignoring, each test generates a Kitfile, a .kitignore and files representing a model kit. It then packs those files, unpacks them to a separate directory, and verifies that all expected files are present (and no unexpected ones are)

For testing kit remove, each test packs one or more modelkits and verifies that kit remove cleans them up as expected

Each test also incidentally tests other commands, e.g. kit tag or kit list as required. Test logging is fairly verbose, so in test failures are relatively easy to diagnose (add the -test.v flag to see all output).

Linked issues

Related: #119, #85