search

cartertemm / agdb

scrape and archive the audiogames.net game database
MIT License
1 stars 2 forks source link

403 error when attempting to run #1

Closed TheQuinbox closed 2 years ago

TheQuinbox commented 2 years ago

Hi, Today I tried to scrape the database, as I don't have a local copy anymore. However, after installing everything in my venv and running:

python3 agdb.py games.json

I get:

requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://audiogames.net/

Did this tool break? Am I missing something? Thanks.

cartertemm commented 2 years ago

This is the fault of the Cloudflare configuration, likely changed following the downtime a month ago. We will have to set required cookies/headers, or better yet let a module like cloudscraper handle it for us.

I'll patch this soon. Thanks for the find!