alaz / legitbot

🤔 Is this Web request from a real search engine🕷 or from an impersonating agent 🕵️‍♀️?
Other
24 stars 11 forks source link

Possible Facebook RADB source issue? #112

Closed alan-gibson closed 1 year ago

alan-gibson commented 1 year ago

We have been seeing errors for Facebook crawlers in the last couple of days. Walking through the code it seems to fail when source is not provided to the Irrc client:

client.query :radb, 'AS32934'
result = client.perform
Connecting to whois.radb.net
Processing AS32934
Executing "!s-*"
Got "F One or more selected sources are unavailable.
"
'!s-*' failed on 'whois.radb.net' (F One or more selected sources are unavailable.). when processing AS32934 for AS32934
No more queries
Closing a connection to whois.radb.net
Queue 0 guard objects
=> {}

Once a source is provided it seems to behave more as expected:

client.query :radb, 'AS32934', source: :radb
result = client.perform
Connecting to whois.radb.net
Processing AS32934
Executing "!sradb"
Got "C
"
Queue new 0 queries
No more queries
Closing a connection to whois.radb.net
Queue 0 guard objects
=> {"AS32934"=>
  {:ipv4=>
    {"AS32934"=>
      ["31.13.24.0/21",
       "31.13.64.0/18",
       "31.13.64.0/19",

Could something have changed with the service?

leo-quimbee commented 1 year ago

I am also seeing the same. Any help would be appreciated.