Closed GMS103 closed 4 months ago
Thank you for reporting this.
I am able to reproduce this problem. Also, I get the same error if I just run
urlopen('https://moreno.perso.math.cnrs.fr/SageMath/test.sage')
.
And I get the same error with Python3.11.4 installed from the python.org package.
I can successfully retrieve that file with the requests package included in the SageMath-10.3 app.
I checked that the app includes the latest version of the certifi package (2024.02.02) while SageMath-10.4b2 is using version 2023.11.17. I tried upgrading certify to version 2023.11.17 in my python.org Python 3.11.4 and I still got the same error. So the Python 3.11.1 in SageMath-10-3.app and Python 3.11.4 from python.org have the same bug, and it appears to be independent of the version of certifi.
Of course the call to urlopen works fine if I add the keyword argument:
context=ssl._create_unverified_context()
What are the versions of Python and certifi in your local build of Sage for which the load command works?
It looks like your Let's Encrypt certificate was created yesterday. I wonder if that has anything to do with this issue.
Below is what I get. About the certificate, I have tried other sites with older certificates with the same result, so I do not think it is an age issue.
% ./sage
┌────────────────────────────────────────────────────────────────────┐
│ SageMath version 10.4.beta2, Release Date: 2024-04-08 │
│ Using Python 3.9.6. Type "help()" for help. │
└────────────────────────────────────────────────────────────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Warning: this is a prerelease version, and it may be unstable. ┃
┗━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┛
sage: pip show certifi
Name: certifi
Version: 2023.11.17
Summary: Python package for providing Mozilla's CA Bundle.
Home-page: https://github.com/certifi/python-certifi
Author: Kenneth Reitz
Author-email: me@kennethreitz.com
License: MPL-2.0
Location: /Users/gms/SageMath/Git/sage/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages
Requires:
Required-by: httpcore, httpx, requests
Note: you may need to restart the kernel to use updated packages.
sage:
I tested whether I could open your url with urllib.request.urlopen using Python 3.12.0 from python.org. It worked fine, using certifi 2023.07.22. Neither the SageMath Python 3.11.1 nor the python.org Python 3.11.4 can open your url with urlopen. I also tried using Python 3.10.12 on Ubuntu 22.04 with certifi 2020.06.20. That also worked fine.
I conclude that this is a bug in multiple versions of Python 3.11. I am not sure what I can do about it. The Sage project is working on updating its Python spkg to 3.12. Once they do that, this problem should be fixed. But that will probably have to wait at least until SageMath 10.4 is released.
Maybe the urllib.requests package could be patched. I don't know what the nature of the bug is, however.
Thank you very much, Marc, for taking the time of looking into this. Do you think it would be worthwhile for me to try to open a Python issue? I could not find anything relevant in https://github.com/python/cpython/issues?q=is%3Aissue+SSLCertVerificationError
I tried one more experiment. I installed Python3.11.8 (the newest point release of 3.11) in place of my 3.11.4. I also installed the latest certifi package (2024.02.02). I then tried to open your url with urllib.requests.urlopen. It worked fine. So it looks like the problem has been fixed in the latest release of Python 3.11. I don't know what more the Python developers could do, once they have fixed the problem in a point release. They can't recall older versions. So I would say that it is not worth the trouble to file an issue with them.
I will experiment with using Python 3.11.8 in the SageMath app. If I can do that without too much trouble I will do another release of the app. I'm sorry that this happened.
Thank you very much once again, Marc, for all the time and energies you devote to this wonderful SageMath for macOS. If this load thing eventually works it will be most useful for many of my students. But please do not be sorry, it is not your fault.
It turned out that updating python to verson 3.11.8 was straightforward. But that did not fix the problem. I also had to patch the sage.misc.remote_file module. I needed to explicitly tell the urlopen command to use the cert chain provided by the certifi package:
cafile = certifi.where()
content = urlopen(req, timeout=1, context=default_context(cafile=cafile))
I now think this is a python bug, but a subtle one. I am not sure how to construct a reproducible demonstration. It seems that sometimes the default ssl context uses certifi, but sometimes it doesn't.
I have published a pre-release of a new version of SageMath-10-3.app which contains the fix. At least it works for me. @GMS103: would you please test it?
I confirm that SageMath 10.3 - v2.3.1-pre Pre-release works, I am most grateful. Strange bug indeed.
Thanks!
The command
gives an
SSLCertVerificationError
instead ofThis command succeeds in SageMath 10.4.beta2 compiled from sources on macOS 12.7.4 with Homebrew. Also, the URL
https://moreno.perso.math.cnrs.fr/SageMath/test.sage
works as expected in a web browser on the same computer. The full message is: