failed to open url with urllib.request

[ivywze]

Troubleshooting

my environment

• Operating system: ios
• Python version: 3.7
• Jupyter notebook or not? [Y/N]: Y
• Which chapter of book?: notes-week-07

my question

I could not open url using urllib.request library

The minimum code (snippet) to reproduce the issue

import urllib.request

with urllib.request.urlopen('http://www.indeed.com/jobs?q=data+scientist&l=') as url:
    source = url.read()

URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1045)>

the efforts you have spent on this issue

I found a slack issue SSL Certification error #334 have same discussion, but I still could not figure it out.

[hupili]

Seems CVA network problem. Do you have the same problem when testing in other places?

[ivywze]

Seems CVA network problem. Do you have the same problem when testing in other places?

Yes, I test at home, the error message is: URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:777)>

[hupili]

It's Ok at my side. It's likely you are behind firewalls.

[ivywze]

It's Ok at my side. It's likely you are behind firewalls.

So...is there anything I can do to make it run?

[hupili]

Since this is network problem, you need to bypass the walls on the pathways. Using a proxy will help. urllib and requests allows to configure a proxy in the parameters. Depending on your network situation, you may find different proxy providers. Just search "HTTP proxy", "SOCKS proxy", etc

[ivywze]

Since this is network problem, you need to bypass the walls on the pathways. Using a proxy will help. urllib and requests allows to configure a proxy in the parameters. Depending on your network situation, you may find different proxy providers. Just search "HTTP proxy", "SOCKS proxy", etc

Thanks for the pointers, will do accordingly.