Open JGuetschow opened 4 years ago
It's not related to 3.5, can reproduce on 3.7.
Captcha when going to https://unfccc.int/process-and-meetings/transparency-and-reporting/reporting-and-review-under-the-convention/greenhouse-gas-inventories-annex-i-parties/national-inventory-submissions-2019 in browser:
Why am I seeing this page?
The website you are visiting is protected and accelerated by Incapsula. Your computer may have been infected by malware and therefore flagged by the Incapsula network. Incapsula displays this page for you to verify that an actual human is the source of the traffic to this site, and not malicious software.
I don't get a captcha. Maybe a cookie that's set can avoid the captcha?
But it looks like they want to prevent mass downloads.
The nightly GitHub Action also seems to have worked fine.
Tried to add cookies (my browser cookies where I get no captcha) to the requests call, but it didn't help. Might have done something wrong though.
As a workaround you could try saving the HTML file from the browser and parsing that in the script.
I've now tried
So, I'm currently out of ideas
I also tried adding a browser user agent ...
Which files are you trying to get?
crf 2019 currently.
The overview page is loaded from a file and then I get the following message
Error fetching https://unfccc.int/documents/195779
It's retrying until I manually interrupt
When using the current version of the downloader with Python 3.5.2 I get an AttributeError:
Python version