search

zilliztech / VectorDBBench

A Benchmark Tool for VectorDB
MIT License
458 stars 108 forks source link

Regarding adding new client #254

Open mithunputhusseri opened 6 months ago

mithunputhusseri commented 6 months ago

Hi Team,

While adding a new client for testing. I have followed all the steps mentioned in the page. Finally I have also made changes in the clients/init.py file. Where I have added to the entries the DB(Enum) class. However when I restart the app the changes are not reflected in the the GUI. And there is no error thrown. Is there any other steps, I have to follow.

Thanks, Mithun

alwayslove2013 commented 6 months ago

@mithunputhusseri Maybe you can try rerunning init_bench.

alwayslove2013 commented 6 months ago

BTW, you need to prepare an icon_url and color for your client in vectordb_bench/frontend/const/styles.py.

https://github.com/zilliztech/VectorDBBench/blob/c5b7b34d8095a5f8cc5be96247731a85609252a0/vectordb_bench/frontend/const/styles.py#L38-L62

mithunputhusseri commented 6 months ago

Hi Team, I have made those changes aswell. But still the changes are not reflected. I have debugged this issue further and found out that the issue is with Streamlit framework. It uses a file cache system. Which cache file changes. It heavily depends on file watchers to read file changes and restart. However somehow this file watcher is buggy and new changes are not getting reflected. Which causes this issue. Streamlit issue page: https://discuss.streamlit.io/t/re-run-refresh-does-not-update-page-anymore-on-1-24-0-need-restart-of-streamlit-for-update-to-happen/46356/7

This is a weird issue and if run streamlit run file.py on individual files new changes get reflected. But branched file changes are not reflected. It would be great if you could somehow turn off file cache for streamlit. Since this part of the framework is buggy and they have not been able to fix it for past several labels.

Thanks, Mithun

alwayslove2013 commented 6 months ago

@mithunputhusseri Thanks so much~ We'll keep an eye on Streamlit.

We wanna know whether your newly added client works now.

Another possibility we found during our testing is that when you have more than one vectordb-bench repo on your computer (e.g. installed one via pip install and another cloned via git), there may be a problem with the installer pointing to the wrong package so that the file changes are not tracked correctly.

If you have two vectordb-bench on your computer, go inside the dir of the right repo and the following code will resolve it to the current directory.

pip install -e ".[test]"
init_bench
mithunputhusseri commented 6 months ago

sure @alwayslove2013 , I will try this out