rapid7 / recog

Pattern recognition for hosts, services, and content
Other
662 stars 195 forks source link

recog_standard crashes when identifier files are missing. #384

Closed tsellers-r7 closed 2 years ago

tsellers-r7 commented 2 years ago

Describe the bug bin/recog_standardize crashes when trying to process a file that does not exist.

Sometimes values are added to the identifier files and then they are removed from the fingerprints later without cleaning up the identifier files. bin/recog_standardize doesn't remove orphaned entries.

If, instead, it were to create missing files then that would make it easy for contributors to ensure the accuracy of files in identifiers/ by simply deleting all of the txt files and running the tool with the -w flag.

Additionally, it would be nice to have a flag that just does this for us for any file that recog_standardize is processing.

To Reproduce

$ rm identifiers/vendor.txt 

$ ruby bin/recog_standardize -w xml/*
Traceback (most recent call last):
    2: from bin/recog_standardize:52:in `<main>'
    1: from bin/recog_standardize:10:in `load_identifiers'
bin/recog_standardize:10:in `readlines': No such file or directory @ rb_sysopen - /Users/something/git/recog/identifiers/vendor.txt (Errno::ENOENT)

Expected behavior recog_standardize creates any missing files

tsellers-r7 commented 2 years ago

For the flag, we could just have it return an empty map here when set

https://github.com/rapid7/recog/blob/54325128c722f8a9890c23092c3e072aeb884b14/bin/recog_standardize#L8-L14

The check for file existence should likely be just above the File.readlines() and do the same thing.