Hashes missing in output file

rjhansen / nsrllookup

Checks with NSRL RDS servers looking for for hash matches

https://rjhansen.github.io/nsrllookup

ISC License

111 stars 10 forks source link

Hashes missing in output file #10

Closed Souravc413 closed 5 years ago

Souravc413 commented 5 years ago

Hi, If I have a text file which has X number of hashes and Im looking them up with nsrllookup, it produces two files, one with the known hashes eg k.txt and another unknown u.txt. However the sum of hashes in both files is always less than X. It happens when i lookup 50 hashes or 5000. I think the RDS hits would be in the known files and the misses will be in the Unknown. It happens both when I use my own instance of nsrlsvr and nsrllookup.com Can you please help me with this?

rjhansen commented 5 years ago

I'm sorry, for some reason I didn't see this issue before.

There is a good reason for the difference in results. If there are multiple copies of a hash in the input set, redundant hashes will be deduplicated.

If you have a set of X unique hashes, and you're getting sets k and u which do not add up to X, then we've got a problem. Can you look into this for me and tell me what's going on?

soham637 commented 5 years ago

@rjhansen Hi Rob, The fault was on my part, there were non unique hashes in the file. Thank you for the guidance.