rapid7 / recog

Pattern recognition for hosts, services, and content
Other
662 stars 195 forks source link

Cross language compatibility: case insensitivity flag #552

Closed TomSellers closed 1 year ago

TomSellers commented 1 year ago

Description

This PR moves the inline flag (?i) (case insensitivity) to the beginning of the regex. In Python this flags has global meaning and use of it anywhere other than the start of the pattern is deprecated. This does not change the function of this flag in current patterns for other languages.

See previous work in https://github.com/rapid7/recog/pull/367

This was caught by internal testing in recog-go and the fixes are required in order for us to implement and test the changes from the following PRs into recog-go:

How Has This Been Tested?

bundle exec rake tests ruby bin/recog_verify xml/*.xml | grep -v WARN | grep -v SUMMARY

Types of changes>

Checklist:

mkienow-r7 commented 1 year ago

This was caught by internal testing in recog-go ...

The hope behind the Go Verify workflow was to detect issues between the various recog language implementation. What sort of internal testing discovered this issue that was not discoverable when verifying the fingerprints against their examples via cmd/recog_verify/main.go?

TomSellers commented 1 year ago

@mkienow-r7

The hope behind the Go Verify workflow was to detect issues between the various recog language implementation. What sort of internal testing discovered this issue that was not discoverable when verifying the fingerprints against their examples via cmd/recog_verify/main.go?

Testing in the Go version of recog_verify should be the same as the ruby version so that we can test consistently.

The test for this condition is in fingerprints_test.go which would similar to the spec tests.

https://github.com/runZeroInc/recog-go/blob/1f868cde9c4b2e110f99d1114d1df4e530842f90/fingerprints_test.go#L51-L58

EDIT: I'll see about adding checks to the Ruby and Go versions recog-verify to avoid XML being Ok with one implementation and not another.