RIPE-NCC / bgpdump

Utility and C Library for parsing MRT files
Other
60 stars 9 forks source link

Too many prefixes #10

Closed HappyTTL closed 10 months ago

HappyTTL commented 2 years ago

When I was running bgpdump I got the Too many prefixes error for some files. For example: (base) root@devcloud:/data/anycast_monitor/routeviews# /usr/bin/bgpdump -m route-views2.updates.20220308.0015.bz2 | head 2022-03-08 08:33:37 [info] logging to syslog

/var/log/messages-20220307:Mar 7 03:17:09 VM-66-201-centos bgpdump: too many prefixes (1851628600 > 88) (base) root@devcloud:/data/anycast_monitor/routeviews# date Tue Mar 8 08:34:37 CST 2022 (base) root@devcloud:/data/anycast_monitor/routeviews# date -u Tue Mar 8 00:34:43 UTC 2022

But it works ok with another file (base) root@devcloud:/data/anycast_monitor/routeviews# /usr/bin/bgpdump -m route-views.saopaulo.updates.20220308.0015.bz2 | head 2022-03-08 08:35:22 [info] logging to syslog BGP4MP|1646698500|A|2001:12f8::223:254|26162|2804:fec:2000::/36|262493|IGP|2001:12f8::212:142|0|0|26162:65011|NAG|| BGP4MP|1646698500|A|2001:12f8::223:254|26162|2804:fec:9000::/36|262493|IGP|2001:12f8::212:142|0|0|26162:65011|NAG|| BGP4MP|1646698501|A|2001:12f8::254|26162|2804:fec:2000::/36|262493|IGP|2001:12f8::212:142|0|0|26162:65011|NAG|| BGP4MP|1646698501|A|2001:12f8::254|26162|2804:fec:9000::/36|262493|IGP|2001:12f8::212:142|0|0|26162:65011|NAG|| BGP4MP|1646698501|A|2001:12f8::218:121|52720|2804:fec:2000::/36|52720 262493|IGP|2001:12f8::218:121|0|0||NAG|| BGP4MP|1646698501|A|2001:12f8::218:121|52720|2804:fec:9000::/36|52720 262493|IGP|2001:12f8::218:121|0|0||NAG|| BGP4MP|1646698501|A|2001:12f8::253|26162|2804:fec:2000::/36|262493|IGP|2001:12f8::212:142|0|0|26162:65011|NAG|| BGP4MP|1646698501|A|2001:12f8::253|26162|2804:fec:9000::/36|262493|IGP|2001:12f8::212:142|0|0|26162:65011|NAG|| BGP4MP|1646698501|A|2001:12f8::223:253|26162|2804:fec:2000::/36|262493|IGP|2001:12f8::212:142|0|0|26162:65011|NAG|| BGP4MP|1646698501|A|2001:12f8::223:253|26162|2804:fec:9000::/36|262493|IGP|2001:12f8::212:142|0|0|26162:65011|NAG||

This is running on a Centos machine. I also noticed that the max prefix is changing, could you help me to understand what happened here?

Also, I tried on another machine with the same two files, there's no such problem. Both of the two files can be parsed.

spakka commented 2 years ago

Hi,

The max prefix stuff has always been a mess. I removed it and rewrote it to use dynamic memory allocation instead of arbitrary limits. Seems that I never merged the fix though. Can you check out: https://github.com/RIPE-NCC/bgpdump/pull/8

And let me know if it works? If so, I will just merge it.

Thanks, Colin

HappyTTL commented 2 years ago

How can I check if it works? Just clone the branch and rebuild?

Colin Petrie @.***> 于2022年3月8日周二 00:55写道:

Hi,

The max prefix stuff has always been a mess. I removed it and rewrote it to use dynamic memory allocation instead of arbitrary limits. Seems that I never merged the fix though. Can you check out:

8 https://github.com/RIPE-NCC/bgpdump/pull/8

And let me know if it works? If so, I will just merge it.

Thanks, Colin

— Reply to this email directly, view it on GitHub https://github.com/RIPE-NCC/bgpdump/issues/10#issuecomment-1061547015, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD3LVD2PNYBJ3RCQKV73IJTU64IXXANCNFSM5QE3XN2A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

-- Regards, Tingting Lu

HappyTTL commented 2 years ago

Previously I was using a pre-built version. I tried to clone the repo and build on my machine, there are too many errors due to different compiler version. Could you please provide me a built binary that I can try?

Tingting Lu @.***> 于2022年3月8日周二 08:58写道:

How can I check if it works? Just clone the branch and rebuild?

Colin Petrie @.***> 于2022年3月8日周二 00:55写道:

Hi,

The max prefix stuff has always been a mess. I removed it and rewrote it to use dynamic memory allocation instead of arbitrary limits. Seems that I never merged the fix though. Can you check out:

8 https://github.com/RIPE-NCC/bgpdump/pull/8

And let me know if it works? If so, I will just merge it.

Thanks, Colin

— Reply to this email directly, view it on GitHub https://github.com/RIPE-NCC/bgpdump/issues/10#issuecomment-1061547015, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD3LVD2PNYBJ3RCQKV73IJTU64IXXANCNFSM5QE3XN2A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

-- Regards, Tingting Lu

-- Regards, Tingting Lu

HappyTTL commented 2 years ago

I made some changes to the code and now it was successfully built on my machine. Successfully tested on a few files already. I will let it run for a while to make sure it works as expected.

Tingting Lu @.***> 于2022年3月8日周二 08:58写道:

How can I check if it works? Just clone the branch and rebuild?

Colin Petrie @.***> 于2022年3月8日周二 00:55写道:

Hi,

The max prefix stuff has always been a mess. I removed it and rewrote it to use dynamic memory allocation instead of arbitrary limits. Seems that I never merged the fix though. Can you check out:

8 https://github.com/RIPE-NCC/bgpdump/pull/8

And let me know if it works? If so, I will just merge it.

Thanks, Colin

— Reply to this email directly, view it on GitHub https://github.com/RIPE-NCC/bgpdump/issues/10#issuecomment-1061547015, or unsubscribe https://github.com/notifications/unsubscribe-auth/AD3LVD2PNYBJ3RCQKV73IJTU64IXXANCNFSM5QE3XN2A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you authored the thread.Message ID: @.***>

-- Regards, Tingting Lu

-- Regards, Tingting Lu

spakka commented 2 years ago

Yes indeed, just clone the branch and run ./bootstrap.sh; ./configure; make; make test, and then use the ./bgpdump binary to test your files. I look forward to hearing if it works properly.

Thanks, Colin

spakka commented 10 months ago

Fixed with PR #8