rapid7 / recog

Pattern recognition for hosts, services, and content
Other
671 stars 199 forks source link

Fix additional warnings due to untested attributes #404

Closed dabdine closed 2 years ago

dabdine commented 2 years ago

Description

Fixes these additional warnings that #400 did not address due to the examples missing content that could be parsed by the regular expression:

xml/http_servers.xml:1225: WARN: 'Sun Java System Application Server (formerly iPlanet Application Server, Sun ONE Application Server)' is missing an example that checks for parameter 'service.version' which is derived from a capture group
xml/operating_system.xml:59: WARN: 'Windows 2000' is missing an example that checks for parameter 'os.version' which is derived from a capture group
xml/operating_system.xml:228: WARN: 'Gentoo Linux' is missing an example that checks for parameter 'os.version' which is derived from a capture group
xml/operating_system.xml:663: WARN: 'IBM OSes' is missing an example that checks for parameter 'os.version' which is derived from a capture group
xml/operating_system.xml:679: WARN: 'HP OSes' is missing an example that checks for parameter 'os.version' which is derived from a capture group
xml/operating_system.xml:693: WARN: 'Juniper' is missing an example that checks for parameter 'os.version' which is derived from a capture group
xml/operating_system.xml:705: WARN: 'Cisco' is missing an example that checks for parameter 'os.version' which is derived from a capture group
xml/sip_user_agents.xml:286: WARN: 'Polycom RealPresence Trio Phones' is missing an example that checks for parameter 'host.mac' which is derived from a capture group
xml/smtp_banners.xml:541: WARN: 'MailEnable - Simple' is missing an example that checks for parameter 'host.name' which is derived from a capture group

Most of these (especially in operating_systems.xml) were made up to satisfy the regular expression, while others were incorporated using real (anonymized) data from internet scan engines.

After this patch, the only warnings that will remain are those that complain about fingerprints which have no test cases (<example> elements).

Motivation and Context

Clean more warnings.

How Has This Been Tested?

bundle exec rake tests

Types of changes

Checklist:

mkienow-r7 commented 2 years ago

Would you please take a look at the made up parameter values and ensure they have provenance?

dabdine commented 2 years ago

The issue I have with operating_systems.xml in particular is that nothing is defined as to what the input should be for this file. Because of that, despite these changes adding more provenance to actual versions, they may not test real world scenarios.

I'll add that the Cisco fingerprint looks entirely broken (for IOS at least) if it is to be applied to the output of show version. The fingerprint was made to be generic, but won't parse complicated Cisco IOS versions like 12.2(33)SXI1. If memory serves, in many of these cases some of the version info was received from SNMP.

mkienow-r7 commented 2 years ago

I appreciate you taking the time to enhance some of the fingerprints and adding more provenance. After spending more time looking at the fingerprints I agree that we need to make more sense of data source that to be used here and reassess this DB. Thank you for the contribution @dabdine!