Hashs number (fingerprints number)

[fbrcosta]

Hi, First of all, let me congratulate you for the great work.

So, i read your explanation about dejavu, and i've been testing your solution. I noticed that the solution can generate a lot of fingerprints (240k for "Mirrors" like you said on the explanation).

I've built a database with 200+ songs, and the average number of fingerprints for each song is 8k (the maximum is 16k). First i thought it was the songs, but i also added the music "Mirrors" to my database, and it only has 10k fingerprints.

Do you know what i might have done wrong? I've used the default params to generate the fingerprints: DEFAULT_AMP_MIN = 10 PEAK_NEIGHBORHOOD_SIZE = 20 MIN_HASH_TIME_DELTA = 0 MAX_HASH_TIME_DELTA = 200

Thank you in advance.

Cumps, Fábio

[worldveil]

The writeup I posted was using different settings - I think I may have optimized them before sending to github.

I wouldn't worry about it unless you are seeing adverse accuracy or recognition effects?

[fbrcosta]

Hi, the thing is that i was expecting better results than the one's i'm getting.

I built a database with 150 songs. From those songs, I've chosen 50 songs randomly, and for each song I've extracted 1 to 10 seconds of audio starting at second 30 (I've used ffmepg to do this: ffmpeg -ss 30 -t $i -i "echo $file" "../echo $file_name30echo $isec.mp3").

the results i had (all in %): sec...no match...invalid match...correct match...accuracy 1............0................20.....................0....................-........ 2...........12...............56....................32..................68...... 3............4................32....................64..................78...... 4............2................10....................88..................81...... 5............0.................6.....................94..................93...... 6............0.................2.....................98..................91...... 7............0.................2.....................98..................91...... 8............0.................2.....................98..................93...... 9............0.................0....................100.................92...... 10..........0.................0....................100.................96......

Since i was using the original files, i was expecting getting 100 % correct match for 5+ seconds, and i only got 100% with 9+seconds. Also i never had 100% accuracy. I was also expecting more correct match % for under 4 seconds.

Since i've got high incorrect match % for under 4 seconds, I thought maybe having more fingerprints will increase the confidence of the results and consequently the correct match %,

Edit: Can you provide the settings you used to generate the massive amount of fingerprints?

[worldveil]

It's very possible that performance can change as the corpus size increases. My initial testing on 45 songs by no means guarantees performance on larger corpora.

I'm looking into this now, could you provide the ffmpeg script you used for testing?

[fbrcosta]

Hi, yes, of course. I used the ffmpeg script only to generate the test files https://dl.dropboxusercontent.com/u/3025156/dejavu-testing/generate-test-files.sh

I've made also a python script to generate the results: https://dl.dropboxusercontent.com/u/3025156/dejavu-testing/result_dejavu.py

But if you this script to generate the results, be careful about songs with special chars. This script is not prepared to handle special chars yet.

[fbrcosta]

Hi

I was able to solve the problem.

The peaks were not ordered before, so i decided to order them before generate the fingerprints. And now it generates a lot more more fingerprints. For the music 'Mirrors', before it only generated 10k fingerprints. Now, it generates 174k fingerprints.

I added the following code in function 'generate_hashes' before the loop:

from operator import itemgetter peaks.sort(key=itemgetter(1))

Cumps, Fabio

[worldveil]

@fbrcosta apologies! I haven't had much time to look into this lately.

I have looked at making the ordering correctly before (due to another issue raised), but hadn't had time to verify the accuracy did not degrade.

Did you verify the accuracy did not degrade after sorting?

If so, I'd be very happy to have you make a pull request and incorporate the change into the master dejavu branch.

[fbrcosta]

No worries!

No, actually the accuracy is better now. But to be sure I made some tests using the same method I used before (database with the same 150 songs, the same test files).

the results i had (all in %): sec...no match...invalid match...correct match...accuracy 1............0...................8....................92..................94...... 2............0...................2....................98.................100..... 3............0...................0...................100.................98...... 4............0...................0...................100.................98...... 5............0...................0...................100................100..... 6............0...................0...................100................100..... 7............0...................0...................100................100..... 8............0...................0...................100................100..... 9............0...................0...................100................100..... 10..........0...................0...................100................100.....

As you can see, now i never get 'no match', and with 1 second i got 92% correct match and 94% accuracy on the results. With 5 or more seconds all the results were correct and accurate.

EDIT: btw, before the database had 1.3kk fingerprints. Now it has almost 21.8kk fingerprints. And I didn't notice any difference in the query times.