search

lastmile-ai / aiconfig

AIConfig is a config-based framework to build generative AI applications.
https://aiconfig.lastmileai.dev
MIT License
962 stars 79 forks source link

Add run-time AiConfig validator for config.load() #291

Open rholinshead opened 12 months ago

rholinshead commented 12 months ago

One thing I've noticed while working on aiconfig tests is that it's super easy to create invalid JSON manually and then the aiconfig runtime just silently ignores it -- for example, I had forgotten to put the model name/settings under a top-level metadata and had to add some console.logs to see why my metadata was null. Programmatic editing and editing via SDK is good, but we may even run into issues there down the line if we update the expected schema contents.

We should have some run-time validation that the loaded aiconfig is valid. To start, can probably just do it in config.load and leverage some schema validation library (e.g., ajv) with our schema definition & error if it's not valid.

Ankush-lastmile commented 12 months ago

The python sdk is backed pydantic which does validation for us. We will need to explore options on ths TS side

jonathanlastmileai commented 11 months ago

I'm going to lower this to mid pri for now so we can focus on 373, 399, 401, and 402.

jonathanlastmileai commented 11 months ago

Agree that a lot of this is done in py by virtue of using pydantic. Not sure about TS equivalent but @rholinshead can take a look when back, hopefully.