search

alaz / legitbot

🤔 Is this Web request from a real search engine🕷 or from an impersonating agent 🕵️‍♀️?
Other
24 stars 11 forks source link

[WIP] Check all addresses for bingbot and adds some tests #14

Closed kirichkov closed 5 years ago

kirichkov commented 5 years ago

This is still work in progress, but a review from your end would be helpful.

The issue I'm facing is that one of the two PTR records po18-218.co2-6nf-srch-2b.ntwk.msn.net doesn't resolve back to any IP, so that would cause in production the bot to hang for a few seconds while the address is resolved, which might result in bingbot considering the site as slow to load, when in fact it isn't

alaz commented 5 years ago

I am sorry for the delay. The change is indeed rather massive and unfortunately it will still be based on the wrong assumption I made back in the days regarding getnameS.

Instead, I have decided to heavily refactor the code. I shall appreciate your opinion on #17

kirichkov commented 5 years ago

I am sorry for the delay. The change is indeed rather massive and unfortunately it will still be based on the wrong assumption I made back in the days regarding getnameS.

No problem at all. I also tried using getnameS as a workaround but I didn't like the outcome either.