readthedocs / readthedocs-docker-images

Docker image definitions used by Read the Docs
115 stars 70 forks source link

Add WIP Dockerfile for urllib3 hotfix #195

Closed agjohnson closed 1 year ago

agjohnson commented 1 year ago

A couple things happening here, but to summarize:

To test / repro (after rebuilding this image):

docker run readthedocs/build:6.0.4 /bin/bash
docs@778c7ce0877f:/$ eval "$(pyenv init -)"
docs@778c7ce0877f:/$ pyenv shell 3.7.9
docs@778c7ce0877f:/$ pip install --quiet requests
WARNING: You are using pip version 20.0.1; however, version 23.1.2 is available.
You should consider upgrading via the '/home/docs/.pyenv/versions/3.7.9/bin/python3.7 -m pip install --upgrade pip' command.
docs@778c7ce0877f:/$ python -m requests
Traceback (most recent call last):
  File "/home/docs/.pyenv/versions/3.7.9/lib/python3.7/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/home/docs/.pyenv/versions/3.7.9/lib/python3.7/runpy.py", line 142, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/home/docs/.pyenv/versions/3.7.9/lib/python3.7/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/home/docs/.pyenv/versions/3.7.9/lib/python3.7/site-packages/requests/__init__.py", line 43, in <module>
    import urllib3
  File "/home/docs/.pyenv/versions/3.7.9/lib/python3.7/site-packages/urllib3/__init__.py", line 39, in <module>
    "urllib3 v2.0 only supports OpenSSL 1.1.1+, currently "
ImportError: urllib3 v2.0 only supports OpenSSL 1.1.1+, currently the 'ssl' module is compiled with OpenSSL 1.0.2n  7 Dec 2017. See: https://github.com/urllib3/urllib3/issues/2168
agjohnson commented 1 year ago

Developments: @humitos noted that we in fact do not have openssl 1.1 headers installed, and we found installing them (package libssl-dev) breaks a lot of things.

We are considering unpacking the package to a separate path, building against that using CFLAGS/LDFLAGS.

Also also, we will have a worse version of this in september, when the urllib3 package drops support for openssl 1.1.1 (ubuntu 20.04)

humitos commented 1 year ago

Developments: @humitos noted that we in fact do not have openssl 1.1 headers installed, and we found installing them (package libssl-dev) breaks a lot of things.

apt update
apt install libssl-dev     # it removes some packages due to conflicts
<compile Python 3.7.x and 3.8.x with the installed libssl-dev>
apt install libssl1.0-dev node-gyp nodejs-dev npm    # re-install the packages that were removed in the previous step

I was able to compile Python with a newer SSL version this way and able to import requests and make a request. I think this could be a good temporal fix for now.

We are considering unpacking the package to a separate path, building against that using CFLAGS/LDFLAGS.

I tried passing different flags and I was not able to build with the openssl that I downloaded manually. We may need to keep researching about this if we want to go this way.

humitos commented 1 year ago

Also also, we will have a worse version of this in september, when the urllib3 package drops support for openssl 1.1.1 (ubuntu 20.04)

https://github.com/urllib3/urllib3/issues/2168

humitos commented 1 year ago

I tested this locally and it worked. The tutorial-template and linkcheck branch from test-builds built successfully with the image that this Dockerfile generates.

I want to be clear on this. I'm not happy with the solution and I strongly think we need to work on deprecating these images and remove the ability for users to use them. They are gonna cause us this type of issues all the time. However, I'm fine moving forward with this as a quick/hotfix for now, but we should keep the conversation active and do the required work to remove this images from our servers 😄

I'm planning to finish the build, upload to Docker Hub, update Salt in -ops and do a hotfix release today if we have enough time. Otherwise, it's gonna be a tomorrow thing.