search

instructlab / sdg

Python library for Synthetic Data Generation
https://pypi.org/project/instructlab-sdg/
Apache License 2.0
24 stars 37 forks source link

Run the simple pipeline on small runners #372

Closed bbrowning closed 1 week ago

bbrowning commented 1 week ago

Playing whackamole to get a working CI on the 0.3 release branch, I missed https://github.com/instructlab/instructlab/commit/431d909b83230abcaea25eff74cc9df39894a0c3 where there's a new param to e2e-custom.sh that was added and we need to run the simple (vs full) pipeline on the small runner, since it uses a quantized merlinite as the teacher model.

bbrowning commented 1 week ago

One more commit (for now, can't guarantee the last) to try and get a working CI on the 0.3 release branch. Unfortunately still don't have a good way to test these before merging, so manually review and then I'll hit merge and see what happens.

bbrowning commented 1 week ago

The e2e here is going to fail, so not going to wait on it and just manually merge to try and get CI green on the release branch.