search

visual-layer / fastdup

fastdup is a powerful free tool designed to rapidly extract valuable insights from your image & video datasets. Assisting you to increase your dataset images & labels quality and reduce your data operations costs at an unparalleled scale.
Other
1.56k stars 76 forks source link

Error: when running with run_mode=4 (NN on stored features) or run_mode=6 (kmeans on stored features) you must provide a test_dir. Run mode=4 #152

Closed kaane8520 closed 1 year ago

kaane8520 commented 1 year ago

Hi, I was trying to use my pre-computed embeddings as described here: https://github.com/visual-layer/fastdup/blob/main/RUN.md#external (and because it was proposed in this issue: https://github.com/visual-layer/fastdup/issues/68 ) Unfortunately I got error while running this line: fastdup.run(os.path.join(work_dir, 'atrain_features.dat.csv'), work_dir=test_dir, d=d ,run_mode=4)

_Error: when running with run_mode=4 (NN on stored features) or run_mode=6 (kmeans on stored features) you must provide a testdir. Run mode=4

obraz_2023-04-19_112747128 obraz_2023-04-19_112828551

Adding _testdir as a parameter pointing to the directory with my test-images gives me another error: 

[ERROR] Failed to read filenames from test_dir/tmp/files0.txt, file is empty or no line found to be image or tar file. Command that failed to generate file list was find -L path_to_my_test_dir -maxdepth 1 -type f -name 'features' | grep '.csv' | grep -v '.bad.csv' | sort > test_dir/tmp/files0.txt. Please make sure the command works in terminal.

Could you help me to proceed with this?

dbickson commented 1 year ago

hi @kaane8520 thanks for reaching out. For run_mode=4, please provide test_dir pointing to the location of your new feature vectors while work_dir pointing to the original work_dir you run with run_mode=2. Try it out and let us know if this works.

kaane8520 commented 1 year ago

Thanks @dbickson, it worked