Test for persist overwrite

[atroyn]

Multiple Chroma clients can stomp each other's persistence.

[albertovilla]

I don't know how do you plan to solve this but a partial solution would involve:

1. Read current content from the existing parquet into a dataframe (e.g. df_parquet)
2. Get content from the database as a dataframe (e.g. ' df_db' )
3. Merge df_parquet and df_db

This is a partial solution as there are some tricky cases to consider:

• Some items have been modified both in df_parquet and df_db. Which one do you keep? The latest?
• What if the parquet file is written after you've read it? i.e. in between you do this persist.

I guess one option would be to try to implement this into some kind of thread-safe singleton class for the persistance i.e. forcing to use a single instance of an object to perform the persists will ensure consistency.

[HammadB]

This was an issue in the duckdb regime, we have tests for this sort of thing now!