alaz / legitbot

🤔 Is this Web request from a real search engine🕷 or from an impersonating agent 🕵️‍♀️?
Other
22 stars 9 forks source link

Possible Facebook RADB source issue? #112

Closed alan-gibson closed 10 months ago

alan-gibson commented 10 months ago

We have been seeing errors for Facebook crawlers in the last couple of days. Walking through the code it seems to fail when source is not provided to the Irrc client:

client.query :radb, 'AS32934'
result = client.perform
Connecting to whois.radb.net
Processing AS32934
Executing "!s-*"
Got "F One or more selected sources are unavailable.
"
'!s-*' failed on 'whois.radb.net' (F One or more selected sources are unavailable.). when processing AS32934 for AS32934
No more queries
Closing a connection to whois.radb.net
Queue 0 guard objects
=> {}

Once a source is provided it seems to behave more as expected:

client.query :radb, 'AS32934', source: :radb
result = client.perform
Connecting to whois.radb.net
Processing AS32934
Executing "!sradb"
Got "C
"
Queue new 0 queries
No more queries
Closing a connection to whois.radb.net
Queue 0 guard objects
=> {"AS32934"=>
  {:ipv4=>
    {"AS32934"=>
      ["31.13.24.0/21",
       "31.13.64.0/18",
       "31.13.64.0/19",

Could something have changed with the service?

leo-quimbee commented 10 months ago

I am also seeing the same. Any help would be appreciated.