search

worldveil / dejavu

Audio fingerprinting and recognition in Python
MIT License
6.45k stars 1.44k forks source link

Trying to fingerprint about 200 000 files. After 15000 files INSERT operation is very slow. #284

Open unbrokendub opened 2 years ago

unbrokendub commented 2 years ago

Hi all. I stuck when trying to fingerpint big database of music. Becouse of the billions of indexes generated, INSERT operation take super big amount of time. In example first 3000 files on emtpy database (i use postgres) been ready in 3 hours. But after 15000 files added another 3000 files takes more than 10 hours, and looks like time will grow exponencialy. Is there any tweaks to hussle with indexes or maybe it is possible to run dejavu without indexes?

Cheers Denis

busterbeam commented 2 years ago

If you don't need to use a database. Then just modify the main code to what you want it to do

https://github.com/worldveil/dejavu/blob/e56a4a221ad204654a191d217f92aebf3f058b62/dejavu/logic/fingerprint.py#L21-L156

the main logic is in the fingerprint.py