abcsys / libem

Compound AI toolchain for fast and accurate entity matching, powered by LLMs.
https://libem.org
Apache License 2.0
19 stars 3 forks source link

Add configuration validation in calibration #38

Open zenodflow opened 4 months ago

zenodflow commented 4 months ago

Configurations may not always be valid. E.g., value out of bound (negative batch size), conflicting or unsupported combinations of configurations (confidence and cot with prompt-level batching) etc.

We should catch them early on -- no later than the time of calibrate calls materialize the values. E.g., adding assertions in the Parameter class.