Open penguineer opened 1 year ago
Confirming the behavior
It can be fixed by modifying the request headers:
req_url = f"https://www.reichelt.de/index.html?ACTION=446&LA=0&nbc=1&q={ keyword.lower().replace(' ', '%20') }"
req_headers = {
'Accept-Language': 'de,en-US;q=0.7,en;q=0.3'
}
website = requests.get(req_url, headers=req_headers).content.decode('utf-8')
I used python requests here but there surely is a way to change headers with urlopen()
When using the tool, I get the following response:
Which is unfortunate, since I have the same problem in my tool and was hoping to find a solution here.
From another debug session, I could extract this response:
Seems that there is some additional protection now?
Curl, on the other hand, works: