search

pierky / arouteserver

A tool to automatically build (and test) feature-rich configurations for BGP route servers.
https://arouteserver.readthedocs.org/
GNU General Public License v3.0
284 stars 46 forks source link

registro.br: parse gracefully, logging but skipping errors #135

Closed jof closed 1 month ago

jof commented 3 months ago

I'm getting some errors in using the registro.br WHOIS DB dumps when they have WHOIS errors.

Sometimes in the ftp://ftp.registro.br/pub/numeracao/origin/nicbr-asn-blk-latest.txt file, there will be some lines like:

23106|--whois err--
23202|--whois err--
26622|--whois err--
27652|--whois err--
27656|--whois err--
27715|--whois err--
28125|--whois err--
28127|--whois err--
28132|--whois err--

This breaks arouteserver when parsing this dump and asserts that there are always at least three fields per row.

It would be ideal if arouteserver would gather whatever prefix/origin mappings it can from a source, even if there are errors in parsing some records. Some up-to-date information is better than no updated information. Instead of failing the entire source for one error, this change just skips over those error-generating lines and doesn't add origination info for them from the dump.

pierky commented 1 month ago

Hello, thanks for your PR! I've just pushed out a new alpha release that contains this change. If CI/CD will be happy, it will be available under the PyPi test instance in a few hours. It will be possible to install it by following the instructions at https://arouteserver.readthedocs.io/en/latest/INSTALLATION.html#development-and-pre-release-versions

If everything goes well, I plan to ship it as part of a new release in a few days.

Thanks again