drbrain / net-http-persistent

Thread-safe persistent connections with Net::HTTP
http://seattlerb.rubyforge.org/net-http-persistent
339 stars 117 forks source link

Does not work over web proxy/firewall? #139

Open mouse07410 opened 2 years ago

mouse07410 commented 2 years ago

I'm trying to use kramdown-rfc from behind a corporate firewall that requires apps to go through the HTTP_PROXY host to reach web sites outside the perimeter.

It looks like net-http-persistent fails to use HTTP_PROXY:

---
- No link definition for link ID 'x' found on line 233
/Users/ur20980/.cache/xml2rfc/reference.RFC.8174.xml: fetching from https://www.rfc-editor.org/refs/bibxml/reference.RFC.8174.xml
^Cmake: *** Deleting file 'draft-cfrg-schwabe-kyber.xml'
Traceback (most recent call last):
Traceback (most recent call last):
  File "/Users/ur20980/src/draft-schwabe-cfrg-kyber/lib/add-note.py", line 8, in <module>
  File "/Users/ur20980/src/draft-schwabe-cfrg-kyber/lib/.venv/bin/xml2rfc", line 8, in <module>
*** Can't get with persistent HTTP: 
    for line in fileinput.input():
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/fileinput.py", line 256, in __next__
    sys.exit(main())
  File "/Users/ur20980/src/draft-schwabe-cfrg-kyber/lib/.venv/lib/python3.10/site-packages/xml2rfc/run.py", line 530, in main
    line = self._readline()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/fileinput.py", line 392, in _readline
    parser = xml2rfc.XmlRfcParser(source,
  File "/Users/ur20980/src/draft-schwabe-cfrg-kyber/lib/.venv/lib/python3.10/site-packages/xml2rfc/parser.py", line 501, in __init__
    return self._readline()
KeyboardInterrupt

This problem is brought up in https://github.com/martinthomson/internet-draft-template/issues/11

Help is appreciated.

dgholz commented 2 years ago

net-http-persistent supports proxies just fine, https://github.com/rubygems/rubygems/blob/f90de8bdafedde5c106963f149a51dddbbd73aee/bundler/lib/bundler/vendor/net-http-persistent/lib/net/http/persistent.rb#L784

It's quite confusing to work out what you're running, I think it's https://github.com/martinthomson/i-d-template/blob/cf4cd05892ae39c481f270cc23a67fa4b54ac2f2/main.mk#L76. There's nothing there that suggests environment variables are being stripped, have you set it via export HTTP_PROXY=https://your.corp.firewall.setting/?

mouse07410 commented 2 years ago

I'm trying to format an IETF RFC (Internet Draft, to be precise), using Martin Thompson's stuff. The platform is MacOS Monterey.

Yes, I have HTTP_PROXY and HTTPS_PROXY set appropriately. Other apps, like wget and curl (and, pretty much, everything else) seem to have no problem going over proxy to access our retrieve stuff from the "outside".