search

ComputerGhost / FaviconFetcher

Scan a webpage for favicons, or just easily download the one you want.
MIT License
5 stars 3 forks source link

The term "fetch" in the UserAgent results in some requests being blocked by some sites #11

Closed kiddailey closed 1 year ago

kiddailey commented 1 year ago

I've discovered that the term "fetch" in the UserAgent string results in requests being blocked by a number of sites. For example, if you use "https://oracle.com" or "https://minecraft.net" in the FetcherTest of the Example project with the default UserAgent of "FaviconFetcher/1.2", the request will never be fulfilled and the app will report an operation timeout. I suspect this is a poor security attempt to block the "Fetch" FTP software.

If you change the userAgent to something like "FaviconFetc/1.2", "FaviconF3tcher/1.2" or "FaviconFletcher/1.2" and try again, the request will complete normally.

Of course, everyone should be changing their user agent anyway, like so:

((FaviconFetcher.HttpSource)fetcher.Source).UserAgent = "My Custom User Agent";

But I think it might be a good idea to change it as I spent hours trying to figure out why some requests timed out my app and the Example app but not in a browser for the same URL. Will submit a pull request.

ComputerGhost commented 1 year ago

I merged your change with "FaviconF3tcher". I'll increase the version number in a bit. Thank you for the fix.