Closed jsirois closed 8 months ago
Light is starting to dawn on the real issue here: https://github.com/pantsbuild/pants/issues/20467#issuecomment-1935197348
I'll add more info or link to it to close this issue out as truly understood and not just papered over.
Ok, #2358 contains a code comment that buttons this up and the issue can remain closed in good conscience.
For posterity, the test rig used inside a gdb python/install/bin/python3.9
session in thread and no thread modes to suss all this out:
import ssl
import sys
import threading
def create_ssl_context():
return ssl.create_default_context()
SSL_CONTEXT = None
def store_ssl_context():
global SSL_CONTEXT
SSL_CONTEXT = create_ssl_context()
def get_ssl_context():
global SSL_CONTEXT
return SSL_CONTEXT
def main():
args = sys.argv[1:]
if args and args[0] == "--no-thread":
print(create_ssl_context())
else:
thread = threading.Thread(target=store_ssl_context)
thread.daemon = True
thread.start()
thread.join()
print(get_ssl_context())
if __name__ == "__main__":
sys.exit(main())
The only other trick was using a custom debug build of PBS with the patch:
diff --git a/cpython-unix/build-openssl-3.0.sh b/cpython-unix/build-openssl-3.0.sh
index 1d1f913..cd88a0a 100755
--- a/cpython-unix/build-openssl-3.0.sh
+++ b/cpython-unix/build-openssl-3.0.sh
@@ -40,6 +40,7 @@ EXTRA_FLAGS="${EXTRA_FLAGS} ${EXTRA_TARGET_CFLAGS}"
/usr/bin/perl ./Configure \
--prefix=/tools/deps \
--libdir=lib \
+ --debug \
${OPENSSL_TARGET} \
no-legacy \
no-shared \
And built via ./build-linux.py --optimizations debug --python cpython-3.9
.
As initially reported here: https://github.com/pantsbuild/pants/issues/20467
A streamlined repro:
The underlying issue here is still unknown. The Python ssl docs make no mention of any special thread considerations but using PBS Python on older Fedora consistently leads to the above issue, which appears to be solved by calling
ssl.create_default_context(...)
from the main application thread. In this case, it is caused from thepex.jobs.execute_parallel
background job spawn thread here: https://github.com/pantsbuild/pex/blob/a32dd36448103570fd6c1b284164334bc68562da/pex/jobs.py#L538-L555