search

instructlab / training

InstructLab Training Library
Apache License 2.0
13 stars 37 forks source link

Add end-to-end automated test workflow #63

Closed russellb closed 1 month ago

russellb commented 3 months ago

Once this library is being consumed by instructlab/instructlab, it would be helpful to automatically test changes to this library in an end-to-end run of the ilab workflow. You can copy this workflow that I added to the sdg repo to do this. It should work with no modifications, though it does say "instructlab-sdg" in one comment.

https://github.com/instructlab/sdg/pull/33

russellb commented 3 months ago

This was reverted in #76. My comment there explains why this is relevant:

Testing the user end-to-end instructlab workflow is the point of this workflow. It will give you an early indication if your changes to the library break the CLI. Sometimes that might be on purpose, but in case it's not, it will let you know before you publish a release.

This is not intended to replace other functional testing aimed at the library more directly, but the full end-to-end workflow is important.

I'm OK if you want to disable it for the moment, but it needs to go back on at some point. A better way to do that is to just edit the workflow to turn it off instead of removing it from the repo.