Closed kelson42 closed 1 month ago
.. but maybe here we face a bigger scraping problem
This looks like a crawling issue, due to something which detected the crawler and prevented it from operating. Nothing we can fix at code level.
Looks like using "Pixel 5" as mobile device is not triggering the "protection".
I believe the favicon taken as illustration is not in a high resolution enough.
I confirm that scraper takes first "icon" and should take into consideration the "sizes" provided to select the 48x48 one or the biggest one (so that we resize from highest res possible, to avoid side effects from downsizing from something too close to 48x48 and having to resample fractions of pixels).
Scraping https://womenshistory.si.edu/ with an extensive set of good favicon/illustrations:
... But warc2zim seems not able to find a good one![image](https://github.com/user-attachments/assets/3977df55-ce0c-4d00-bd66-e55625b48831)
I suspect it does not looks this new favicon entries
Might be related to https://github.com/openzim/warc2zim/issues/120