search

oughtinc / ice

Interactive Composition Explorer: a debugger for compositional language model programs
https://ice.ought.org
MIT License
529 stars 66 forks source link

Specify tracing file format/API #304

Open alexkreidler opened 1 year ago

alexkreidler commented 1 year ago

ICE is a great tool with a nice UI, and it would be awesome if it could integrate with popular tools like langchain. To do this we need to describe the data structure/types of the JSON file formats in trace.py. One way to do this would be to make the Trace class inherit from the pydantic BaseModel, and then rather than using a static file handler with FastAPI for traces you could set the response_model=List[Trace] which would autogenerate OpenAPI docs that include the trace data structure. But it looks like there's a lot of custom code in trace.py that I don't fully get so that may be tough.

Even just running ICE and getting an example trace, and putting it in the repo/documentation, and maybe passing it through Quicktype to generate a JSON schema from the JSON would help other people use this tool. Thanks!

stuhlmueller commented 1 year ago

Agree that this makes sense - would be happy for someone to take a stab at this!