search

devanshbatham / ParamSpider

Mining URLs from dark corners of Web Archives for bug hunting/fuzzing/further probing
MIT License
2.34k stars 403 forks source link

Error fetching URL #108

Open icssManish opened 7 months ago

icssManish commented 7 months ago

[INFO] Fetching URLs for testphp.vulnweb.com Error fetching URL https://web.archive.org/cdx/search/cdx?url=testphp.vulnweb.com/*&output=txt&collapse=urlkey&fl=original&page=/. Retrying in 5 seconds... Error fetching URL https://web.archive.org/cdx/search/cdx?url=testphp.vulnweb.com/*&output=txt&collapse=urlkey&fl=original&page=/. Retrying in 5 seconds... Error fetching URL https://web.archive.org/cdx/search/cdx?url=testphp.vulnweb.com/*&output=txt&collapse=urlkey&fl=original&page=/. Retrying in 5 seconds... Failed to fetch URL https://web.archive.org/cdx/search/cdx?url=testphp.vulnweb.com/*&output=txt&collapse=urlkey&fl=original&page=/ after 3 retries.

kucingHitam12 commented 7 months ago

i had the same problem [INFO] Fetching URLs for hackerone.com Error fetching URL https://web.archive.org/cdx/search/cdx?url=hackerone.com/*&output=txt&collapse=urlkey&fl=original&page=/. Retrying in 5 seconds... ^CTraceback (most recent call last): File "/home/kerass/.local/lib/python3.11/site-packages/paramspider/client.py", line 57, in fetch_url_content response.raise_for_status() File "/usr/local/lib/python3.11/dist-packages/requests/models.py", line 941, in raise_for_status raise HTTPError(http_error_msg, response=self) requests.exceptions.HTTPError: 503 Server Error: Service Temporarily Unavailable for url: https://web.archive.org/cdx/search/cdx?url=hackerone.com/*&output=txt&collapse=urlkey&fl=original&page=/

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/local/bin/paramspider", line 8, in sys.exit(main()) ^^^^^^ File "/home/kerass/.local/lib/python3.11/site-packages/paramspider/main.py", line 161, in main fetch_and_clean_urls(domain, extensions, args.stream, args.proxy, args.placeholder) File "/home/kerass/.local/lib/python3.11/site-packages/paramspider/main.py", line 95, in fetch_and_clean_urls response = client.fetch_url_content(wayback_uri,proxy) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/kerass/.local/lib/python3.11/site-packages/paramspider/client.py", line 61, in fetch_url_content time.sleep(5) KeyboardInterrupt