Closed TanGentleman closed 2 months ago
Should be fixing a lot of these incongruencies in the coming patch(es)
I decided that this should be modified even before the vectorstore is loaded. This should now be performed whenever the Config class is instantiated.
If I have a vectorstore that was indexed in a prior run, then loading it by reading the collection_name in manifest.json works like a charm.
However, this overrides chunk_size and anything involving in document splitting, since the docs have been indexed in the past. To resolve the mismatch, if a vectorstore is loaded, it should update rag_settings with the important fields (chunk_size, chunk_overlap, embedding model). There should be some clear print statements or warnings when there's an incongruency.