Moar and better unit tests! - Githubissues

alat-rights / alana-utilities

Quick utilities for interacting with LLMs.

https://utils.alana.computer

MIT License

1 stars 2 forks source link

Moar and better unit tests! #15

Open alat-rights opened 2 months ago

alat-rights commented 2 months ago

Top priorities:

Get better test coverage across the API, including for alias functions.
Write more tests, especially for edge cases. Probably the right mentality here is: every parameter in every function is another thing to test.
Test the whole API on a Google Colab, including using different parameters/features

Secondary priorities:

Set up auto-run tests via GitHub Actions.
Have good handling for Anthropic Overloaded Error across all the tests.
If not cost prohibitive, I'd like to test for build errors on every push (instead of every release) on the GitHub Actions side.

Stretch:

Eventually, I'd also like to write regression tests for all the tests in globals. Both simple screens for typos and grammatical errors (possibly via a small Anthropic model) and performance tests on small curated benchmark tasks.

alat-rights commented 2 months ago

I'd like to set up a simple, preferably automated way to run all the simple_tests tests in a Google Colab notebook (or at least a Jupyter notebook of some kind)