search

zmap / zannotate

Utility for annotating Internet datasets with contextual metadata (e.g., origin AS, MaxMind GeoIP2, reverse DNS, and WHOIS)
Apache License 2.0
95 stars 17 forks source link

Fix IP address parse error via trim of trailing newline character w/in AnnotateRead #13

Closed andrewsardone closed 6 years ago

andrewsardone commented 6 years ago

With the merge of https://github.com/zmap/zannotate/pull/12, we changed how AnnotateRead reads in its input with the side-effect of including a trailing \n character. This results in an error down stream when trying to parse the IP address, causing our pipeline to error with this fatal message:

time="2018-04-10T08:00:52-04:00" level=fatal msg="invalid IP received: 104.70.48.35\n"

Note, there might be some better idioms for this string read, but for now this commit is just trying to address the error.

Notes & Caveats

I have not run this code locally, and there is no test coverage.

zakird commented 6 years ago

Thanks for catching this Andrew.

andrewsardone commented 6 years ago

Thanks for catching this Andrew.

👍 @zakird; it was a group effort. @flav pinpointed the failure within the celery tasks, and @cdzombak correlated the 22:05 EDT merge of https://github.com/zmap/zannotate/pull/12 with celery failures starting at 22:08 EDT.