search

httpie / cli

🥧 HTTPie CLI — modern, user-friendly command-line HTTP client for the API era. JSON support, colors, sessions, downloads, plugins & more.
https://httpie.io
BSD 3-Clause "New" or "Revised" License
33.92k stars 3.68k forks source link

HTTPie breaks after connecting over AWS Client VPN #1481

Closed marcomele closed 5 months ago

marcomele commented 1 year ago

Checklist

Minimal reproduction code and steps

  1. Verify http is working normally
  2. Connect to AWS Client VPN
  3. Make any http call to any resource inside the VPC
  4. HTTPie fails to establish the connection (see below)
  5. From this moment forward, any tool using python urllib3 will also fail the same way—e.g. pip can no longer download packages; aws CLI commands start failing: 
    
aws sts get-caller-identity

Could not connect to the endpoint URL: "https://sts.eu-west-1.amazonaws.com/"

Running `aws` with `--debug` shows error cause to be the same as HTTPie, but the behavior only happens after the first HTTPie call.

## Current result
First call to a new host inside VPC (hangs for an average of 16 seconds before failing)

http https:///actuator/health

http: LogLevel.ERROR: ConnectionError: ('Connection aborted.', OSError(9, 'Bad file descriptor')) while doing a GET request to URL: https:///actuator/health

Subsequent calls to the same VPC host, or any other public host after the previous call

http https:///actuator/health

http: LogLevel.ERROR: ConnectionError: HTTPSConnectionPool(host='', port=443): Max retries exceeded with url: /actuator/health (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x10343ed10>: Failed to establish a new connection: [Errno 9] Bad file descriptor')) while doing a GET request to URL: https:///actuator/health

http https://example.com

http: LogLevel.ERROR: ConnectionError: HTTPSConnectionPool(host='example.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x101afc350>: Failed to establish a new connection: [Errno 9] Bad file descriptor')) while doing a GET request to URL: https://example.com/


## Debug output

```bash
http https://example.com --debug
HTTPie 3.2.1
Requests 2.28.1
Pygments 2.13.0
Python 3.11.2 (main, Feb 16 2023, 02:55:59) [Clang 14.0.0 (clang-1400.0.29.202)]
/opt/homebrew/Cellar/httpie/3.2.1/libexec/bin/python3.11
Darwin 22.2.0

<Environment {'apply_warnings_filter': <function Environment.apply_warnings_filter at 0x103d2f240>,
 'args': Namespace(),
 'as_silent': <function Environment.as_silent at 0x103d2f100>,
 'colors': 256,
 'config': {'default_options': []},
 'config_dir': PosixPath('/Users/marco/.config/httpie'),
 'devnull': <property object at 0x103d16d90>,
 'is_windows': False,
 'log_error': <function Environment.log_error at 0x103d2f1a0>,
 'program_name': 'http',
 'quiet': 0,
 'rich_console': <functools.cached_property object at 0x103d310d0>,
 'rich_error_console': <functools.cached_property object at 0x103d311d0>,
 'show_displays': True,
 'stderr': <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>,
 'stderr_isatty': True,
 'stdin': <_io.TextIOWrapper name='<stdin>' mode='r' encoding='utf-8'>,
 'stdin_encoding': 'utf-8',
 'stdin_isatty': True,
 'stdout': <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>,
 'stdout_encoding': 'utf-8',
 'stdout_isatty': True}>

<PluginManager {'adapters': [],
 'auth': [<class 'httpie.plugins.builtin.BasicAuthPlugin'>,
          <class 'httpie.plugins.builtin.DigestAuthPlugin'>,
          <class 'httpie.plugins.builtin.BearerAuthPlugin'>],
 'converters': [],
 'formatters': [<class 'httpie.output.formatters.headers.HeadersFormatter'>,
                <class 'httpie.output.formatters.json.JSONFormatter'>,
                <class 'httpie.output.formatters.xml.XMLFormatter'>,
                <class 'httpie.output.formatters.colors.ColorFormatter'>]}>

>>> requests.request(**{'auth': None,
 'data': RequestJSONDataDict(),
 'headers': <HTTPHeadersDict('User-Agent': b'HTTPie/3.2.1')>,
 'method': 'get',
 'params': <generator object MultiValueOrderedDict.items at 0x10294f540>,
 'url': 'https://example.com'})

http: LogLevel.ERROR: ConnectionError: HTTPSConnectionPool(host='example.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x102aef210>: Failed to establish a new connection: [Errno 9] Bad file descriptor')) while doing a GET request to URL: https://example.com/

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/urllib3/connection.py", line 174, in _new_conn
    conn = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/urllib3/util/connection.py", line 95, in create_connection
    raise err
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
    sock.connect(sa)
OSError: [Errno 9] Bad file descriptor

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
                       ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/urllib3/connectionpool.py", line 386, in _make_request
    self._validate_conn(conn)
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1042, in _validate_conn
    conn.connect()
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/urllib3/connection.py", line 358, in connect
    self.sock = conn = self._new_conn()
                       ^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x102aef210>: Failed to establish a new connection: [Errno 9] Bad file descriptor

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='example.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x102aef210>: Failed to establish a new connection: [Errno 9] Bad file descriptor'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/bin/http", line 33, in <module>
    sys.exit(load_entry_point('httpie==3.2.1', 'console_scripts', 'http')())
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/httpie/__main__.py", line 9, in main
    exit_status = main()
                  ^^^^^^
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/httpie/core.py", line 162, in main
    return raw_main(
           ^^^^^^^^^
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/httpie/core.py", line 136, in raw_main
    handle_generic_error(propagated_exc, annotation=annotation)
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/httpie/core.py", line 100, in raw_main
    exit_status = main_program(
                  ^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/httpie/core.py", line 213, in program
    for message in messages:
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/httpie/client.py", line 111, in collect_messages
    response = requests_session.send(
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/httpie/3.2.1/libexec/lib/python3.11/site-packages/requests/adapters.py", line 565, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='example.com', port=443): Max retries exceeded with url: / (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x102aef210>: Failed to establish a new connection: [Errno 9] Bad file descriptor'))

Additional information, screenshots, or code examples

Behavior started only 2 days ago. No recollection of system, python, or HTTPie updates at the time the issue began. Re-installations do not solve the same issue. Restarting the machine solves the issue until another HTTPie call to a private host is made. Hosts have different network frameworks and application stacks. Other HTTP tools like curl, Postman, and browsers continue to work properly, as well as the HTTPie Desktop app.

wget also fails:

wget -v https://<redacted>/quadrata/passports --debug
DEBUG output created by Wget 1.21.3 on darwin22.1.0.

Reading HSTS entries from /Users/<redacted>/.wget-hsts
URI encoding = ‘UTF-8’
Converted file name 'passports' (UTF-8) -> 'passports' (UTF-8)
--2023-02-20 11:37:45--  https://<redacted>/quadrata/passports
Resolving <redacted> (<redacted>)... 10.0.1.61, 10.0.0.59
Caching <redacted> => 10.0.1.61 10.0.0.59
Connecting to <redacted> (<redacted>)|10.0.1.61|:443... Closed fd 6
failed: Bad file descriptor.
Connecting to <redacted> (<redacted>)|10.0.0.59|:443... Closed fd 6
failed: Bad file descriptor.
Releasing 0x0000600001cb0e00 (new refcount 1).

System

Darwin ip-192-168-1-69.eu-west-1.compute.internal 22.2.0 Darwin Kernel Version 22.2.0: Fri Nov 11 02:04:44 PST 2022; root:xnu-8792.61.2~4/RELEASE_ARM64_T8103 arm64

Python 3.11.2 pip 22.3.1

Ousret commented 5 months ago

As you mentioned yourself: any tool using python urllib3 will also fail the same way It is most likely a OS configuration error rather than a bug in urllib3.