search

devanshbatham / ParamSpider

Mining URLs from dark corners of Web Archives for bug hunting/fuzzing/further probing
MIT License
2.48k stars 422 forks source link

error fetching url #110

Open Ammitte opened 10 months ago

Ammitte commented 10 months ago

[INFO] Fetching URLs for something.com Error fetching URL https://web.archive.org/cdx/search/cdx?url=something.com/*&output=txt&collapse=urlkey&fl=original&page=/. Retrying in 5 seconds... Error fetching URL https://web.archive.org/cdx/search/cdx?url=something.com/*&output=txt&collapse=urlkey&fl=original&page=/. Retrying in 5 seconds... Error fetching URL https://web.archive.org/cdx/search/cdx?url=something.com/*&output=txt&collapse=urlkey&fl=original&page=/. Retrying in 5 seconds... Failed to fetch URL https://web.archive.org/cdx/search/cdx?url=something.com/*&output=txt&collapse=urlkey&fl=original&page=/ after 3 retries.

knrredhelmet commented 2 weeks ago

i got the same problem and when i saw 7 pull requests to this project and still author not approve that so i create a similar tool and solve most of the issue https://github.com/knrredhelmet/paraminer