search

pyocd / cmsis-pack-manager

A Rust and Python module for handling CMSIS Pack files
Apache License 2.0
28 stars 35 forks source link

Add HTTP proxy support #101

Closed flit closed 4 years ago

flit commented 5 years ago

This is a request to add support for the http_proxy environment variable so that the index and packs can be downloaded through a proxy server.

See below from @jeromecoutant in mbedmicro/pyOCD#638. The gist is that curl is able to access the Keil index files from a corporate network behind a proxy, where CPM cannot (it fails with a DNS error).

Hi Back to this issue... I agree that I have some network/PC restrictions due to company policy... ``` $ pyocd pack --install stm32f303 0000348:INFO:__main__:No pack index present, downloading now... May 03 11:43:05.522 ERRO No such host is known. (os error 11001), uri: http://www.keil.com/pack/keil.vidx May 03 11:43:05.523 ERRO No such host is known. (os error 11001), uri: http://www.keil.com/pack/keil.pidx ``` OK ``` $ curl -L http://www.keil.com/pack/keil.vidx -O % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 5668 100 5668 0 0 21308 0 --:--:-- --:--:-- --:--:-- 21308 100 9622 100 9622 0 0 17590 0 --:--:-- --:--:-- --:--:-- 17590 $ curl -L http://www.keil.com/pack/keil.pidx -O % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 5672 100 5672 0 0 60340 0 --:--:-- --:--:-- --:--:-- 60340 100 14216 100 14216 0 0 19771 0 --:--:-- --:--:-- --:--:-- 19771 ``` Seems that curl is calling directly GET HTTP function, whereas pack manager use first DNS call without any proxy setting. All commands using http_proxy environment variable are working (git clone, curl, …) So it seems that restrictions I have could be avoid ?
jeromecoutant commented 5 years ago

@rgf97

theotherjimmy commented 5 years ago

Good debugging @jeromecoutant Thanks for all the detail. I actually use a GET from my API, I just need to figure out how to let the OS do the DNS (like everyone else...)

rmu75 commented 5 years ago

The issue is not the DNS, the issue is not using the proxy. In my environment, DNS resolution of www.keil.com works just fine, but http requests need to go through the proxy. pyocd pack --install fails with "Connection refused (os error 111)".

flit commented 5 years ago

There are probably multiple proxy-related issues here, depending on the network setup. The error reported by @jeromecoutant was "No such host is known. (os error 11001)", which indicates a DNS failure.

rmu75 commented 5 years ago

There are probably multiple proxy-related issues here, depending on the network setup.

I don't think so. In the error reported by @jeromecoutant the proxy is perfectly reachable and hostname of proxy resolves, so there are no DNS issues.

I'm not really fluent in rust and don't know the library landscape, so I can't really check in the source, nevertheless, strace(1)ing pyocd suggest http_proxy / https_proxy environment variables are ignored, iow no proxy functionality is implemented.

theotherjimmy commented 5 years ago

@jeromecoutant @rmu75 Would you be willing to run a test build from my branch https://github.com/theotherjimmy/cmsis-pack-manager/tree/use-reqwest to confirm proxy support in your setups?

rmu75 commented 5 years ago

Sorry for the late reply, very busy atm. Of course I am willing to test this. Thanks.

flit commented 4 years ago

Closing this issue, it should be fixed. CPM now uses reqwest, and calls .use_sys_proxy().

If someone encounters the problem again, please reopen.