search

MartinoMensio / claimreview-collector

dataset processing part of https://github.com/MartinoMensio/MisinfoMe
1 stars 0 forks source link

Archive.today not allowing scraping #8

Closed MartinoMensio closed 1 year ago

MartinoMensio commented 1 year ago

With normal requests, you get HTTP 429. With Flaresolverr, timeout error.

See https://github.com/wabarc/archive.is#archivetoday-is-unavailable as it is probably related.

Affects archive.vn, archive.is, ... (many domains)

MartinoMensio commented 1 year ago

Solved