Closed bluikko closed 2 years ago
Seems that the performance of both (?!) rr.ntt.net
and rr1.ntt.net
has tanked - even a timeout of 360 seconds is not enough.
It is interesting that both of the servers show the same behavior. I wonder if there's some issue that affects both of the servers.
As a test I increased bgpq3_timeout
to 1200 and even that is not enough.
The servers must have issues.
While looking at this I noticed that the arg -R 32
is passed to bgpq4
when retrieving v4 prefixes. I wonder if this might be a place for optimization and instead use cfg.filtering.ipv4_pref_len.max
for -R
? Since prefixes longer than that would be not allowed anyways? And similarly for IPv6 prefixes. (note: I know nothing about how this works internally in arouteserver so maybe this is a bad idea)
Edit: finally it completed, with:
INFO Enricher 'IRRdb prefixes' completed successfully after 1395 seconds
Are you using bgpq3
or bgpq4
?
Are you using
bgpq3
orbgpq4
?
It is the default bgpq of arouteserver docker image. It should be bgpq4
since one week ago it used to take less than a minute.
Not exactly addressing your question, but have you considered running your own IRRd instance? http://irrd.net/ - that way you create a local copy of the IRR data that you can hammer
Not exactly addressing your question, but have you considered running your own IRRd instance? https://irrd.net/ - that way you create a local copy of the IRR data that you can hammer
Yes, every time there is an issue with the default servers... but right now do not have 300 GB of SSD space available.
Hi @bluikko, thanks for the details here.
Not sure that using a backoff value to increase the timeout when different servers are used would help much. Ideally, when a different server is attempted after a previous one failed, the original value of timeout should be enough to answer the query. If the issue is within the server itself, switching to a new one should be the solution, and having a longer timeout should be irrelevant. Of course, I see how the reality may be different than that 😉
Also, in this case the backoff multiplier wouldn't have helped to reach the final target value of 1400 seconds (unless we used a ~12x multiplier, which seems a bit aggressive 😆).
I'll look into the possible optimisation of changing the -R 32
, but however I suspect it wouldn't help in similar circumstances.
@pierky True. Do the servers work for you?
The NTT IRR server performance seems to be back to usual:
INFO Enricher 'IRRdb prefixes' completed successfully after 245 seconds
Hello @bluikko,
thanks for the update.
I've tried to run bgpq4
using -R 32
(as it is in ARouteServer) and also using -R 24
, and I've noticed the same time is needed to complete the query. Given the likely absence of performance optimisation given by a -R <max_pref_len>
change I'm inclined towards keeping the current implementation as it is for the time being.
I'm going to close the ticket, but please feel free to reopen it should you have any further concerns or comments on this regard. Thanks again.
It looks like the performance of
rr.ntt.net
/rr1.ntt.net
varies. Earlier there was no problem looking up the 2 very large AS-SETs AS-HURRICANE/AS-HURRICANEV6 but now they time out. It even had a margin that looked plentiful - but this proves the margin insufficient.Other queries OK.
To be on the safe side, suggest to increase bgpq timeout 200%.
Question: can(nevermind, mounting a customizedbgpq3_timeout
be set in an environment variable? That would be easy for Docker arouteserver.arouteserver.yml
is just as easy)Edit: another option - set an exponential timeout if multiple whois servers?
bgpq3_timeout
.bgpq3_timeout * 2
.bgpq3_timeout * 4