search

snipsco / snips-nlu

Snips Python library to extract meaning from text
https://snips-nlu.readthedocs.io
Apache License 2.0
3.89k stars 513 forks source link

How to download resources for offline use behind a proxy? #859

Closed Moon-developer closed 4 years ago

Moon-developer commented 4 years ago

Question

Hi, I run the following:
python -m snips-nlu download en and snips-nlu download en and I always get the following

requests.exceptions.SSLError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /snipsco/snips-nlu-language-resources/master/shortcuts.json (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1076)')))

Whether I set a proxy or not in my env I still get this this error. Is it possible to add argument flags to manually pass these in during download and whether or not to set verify=True or verify=False in the requests?

Moon-developer commented 4 years ago

For anyone who might be interested in downloading resources for offline or behind a proxy, but don't know where to look, just follow the following instructions:

Goto site-packages/snips_nlu/cli/ and edit the utils.py as follows:

If you are using a corporate proxy edit line 72 and change it r = requests.ge(url, verify=False) to what best works for you behind your proxy. I had to change my verify to True in my case but might be different for you.

Next step is goto line 103 and change pip_args = ['--no-cache-dir', '--no-deps'] to pip_args = ['--no-cache-dir', '--no-deps', '--trusted-host', 'pypi.org', '--trusted-host', 'files.pythonhosted.org', '--proxy=http://your.proxy.com:8080'], Alternatively you can put a print(download_url) in line 103 instead and run the python -m snips_nlu download en command and see the link it gives you. Click it to download the file, extract it, move it to where you want to install it and run the python setup.py install command inside the directory and then run the python -m snips_nlu download en command again and you will see it found the required dependencies and ends up linking it instead of trying to download it.

Hope this helps. Took me awhile to figure this out myself.

adrienball commented 4 years ago

Thanks @Moondog360 I would add that you can pass extra pip args directly to the snips-nlu CLI after --:

python -m snips_nlu download en -- \
    --trusted-host pypi.org \
    --trusted-host files.pythonhosted.org \
    --proxy=http://your.proxy.com:8080
Moon-developer commented 4 years ago

@adrienball I did not find anything to execute it the way you mentioned. This helps a lot. Still needed to changed verify to False on line 72 of utils.py but this still helped a lot working behind corporate proxies.