Describe the bug
The cleanup for the forward_local context manager fails if using requests within the context manager and an exception is raised from requests. Even if I catch the exception within the context manager, and the context manager returns normally, a ThreadException is raised in the context managers cleanup. This then leaves the forwarded port in use such that it is not possible to use it again.
To Reproduce
Use this code to create a tunnel to a server with a self signed certificate. The requests request will fail to validate the certificate and raise an exception. This exception is caught, the execution continues and then a new exception is raised in the context manager cleanup. The port is never released.
import fabric
import requests
from time import sleep
sshargs = {
"host": "192.168.0.195",
"user": "root",
"connect_kwargs": {
"password": "root"
}
}
local_port = 32345
with fabric.Connection(**sshargs).forward_local(local_port, remote_port=443):
sleep(1) # Wait for the tunnel to start working
try:
auth = requests.auth.HTTPDigestAuth(sshargs["user"], sshargs["connect_kwargs"]["password"])
response = requests.get(
f"https://127.0.0.1:{local_port}", auth=auth, verify=True
)
print("RESPONSE: ", response)
print(response.text)
except requests.exceptions.SSLError as e:
print("GOT ERROR: ", e)
print("Done")
The output of the command is:
GOT ERROR: HTTPSConnectionPool(host='127.0.0.1', port=32345): Max retries exceeded with url: / (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1007)')))
Done
Traceback (most recent call last):
File ~/.pyenv/versions/3.10.13/lib/python3.10/site-packages/spyder_kernels/py3compat.py:356 in compat_exec
exec(code, globals, locals)
File ~/.config/spyder-py3/temp.py:19
with fabric.Connection(**sshargs).forward_local(local_port, remote_port=443):
File ~/.pyenv/versions/3.10.13/lib/python3.10/contextlib.py:142 in __exit__
next(self.gen)
File ~/.pyenv/versions/3.10.13/lib/python3.10/site-packages/fabric/connection.py:1003 in forward_local
raise wrapper.value
File ~/.pyenv/versions/3.10.13/lib/python3.10/site-packages/invoke/util.py:209 in run
self._run() # type: ignore
File ~/.pyenv/versions/3.10.13/lib/python3.10/site-packages/fabric/tunnels.py:102 in _run
raise ThreadException(exceptions)
ThreadException:
Saw 1 exceptions within threads (ConnectionResetError):
Thread args: {}
Traceback (most recent call last):
File "/home/danielfa/.pyenv/versions/3.10.13/lib/python3.10/site-packages/invoke/util.py", line 209, in run
self._run() # type: ignore
File "/home/danielfa/.pyenv/versions/3.10.13/lib/python3.10/site-packages/fabric/tunnels.py", line 130, in _run
empty_sock = self.read_and_write(
File "/home/danielfa/.pyenv/versions/3.10.13/lib/python3.10/site-packages/fabric/tunnels.py", line 151, in read_and_write
data = reader.recv(chunk_size)
ConnectionResetError: [Errno 104] Connection reset by peer
Note how "Done" is printed before the exception, so the exception is in the context manager code.
The next time I try to use the port in the same way, I get OSError: [Errno 98] Address already in use.
This does not seem to happen every time, but at least half of the times.
Expected behaviour
There should not be an exception raised from the cleanup
Regardless of any errors, the port should be released
Environment
I don't think the issue is specific to a narrow environment as we have seen this on multiple Linux environments, but here's what I used to replicate it now:
Linux danielfa-ThinkPad-T590 6.2.0-36-generic #37~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct 9 15:34:04 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
Describe the bug The cleanup for the
forward_local
context manager fails if using requests within the context manager and an exception is raised from requests. Even if I catch the exception within the context manager, and the context manager returns normally, aThreadException
is raised in the context managers cleanup. This then leaves the forwarded port in use such that it is not possible to use it again.To Reproduce Use this code to create a tunnel to a server with a self signed certificate. The
requests
request will fail to validate the certificate and raise an exception. This exception is caught, the execution continues and then a new exception is raised in the context manager cleanup. The port is never released.The output of the command is:
Note how "Done" is printed before the exception, so the exception is in the context manager code.
The next time I try to use the port in the same way, I get
OSError: [Errno 98] Address already in use
.This does not seem to happen every time, but at least half of the times.
Expected behaviour
Environment I don't think the issue is specific to a narrow environment as we have seen this on multiple Linux environments, but here's what I used to replicate it now:
Linux danielfa-ThinkPad-T590 6.2.0-36-generic #37~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Mon Oct 9 15:34:04 UTC 2 x86_64 x86_64 x86_64 GNU/Linux
Python libraries: