gordonwatts / snowmass-chat

Experiments exploring the US Snowmass Process documents using LLM
Apache License 2.0
2 stars 0 forks source link

Vector store file should remember how many text chunks there are #7

Open gordonwatts opened 11 months ago

gordonwatts commented 11 months ago

It would be nice to be able to do chatter -c <config-file> vector list to get a listing of not just the references loaded in, but also how many text chunks each has. This would let you know it had extracted the data from the PDF correctly, for example. A nice sanity check.

Currently the place that info is cached and where it is determined are a long ways from each other. Probably requires some sort of callback from model.py, unfortunately.