selwin / python-user-agents

A Python library that provides an easy way to identify devices like mobile phones, tablets and their capabilities by parsing (browser) user agent strings.
MIT License
1.43k stars 196 forks source link

Spider/Crawler Google Favicon not detected as Spider #120

Open ghost opened 1 year ago

ghost commented 1 year ago

Google Favicon is the user agent for downloading favicons defined by websites. https://developers.google.com/search/docs/advanced/appearance/favicon-in-search#crawler

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko)  Chrome/49.0.2623.75 Safari/537.36 Google Favicon