Open daisieh opened 8 months ago
It's possible that this is a Docker thing...?
I created a duplicate issue several weeks ago. This happens with other remote locations too (google cloud and https bam files): https://github.com/pysam-developers/pysam/issues/1268
Thanks for the report. Increasing the verbosity helps identify the problem:
>>> import pysam
>>> pysam.set_verbosity(9)
3
>>> pysam.AlignmentFile('s3://example/foo.bam')
[…]
* Trying 3.5.7.133...
* TCP_NODELAY set
* Connected to example.s3.amazonaws.com (3.5.7.133) port 443 (#0)
* ALPN, offering http/1.1
* error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
[E::easy_errno] Libcurl reported error 77 (Problem with the SSL CA cert (path? access rights?))
[E::hts_open_format] Failed to open file "s3://example/foo.bam" : Input/output error
[…]
This /etc/pki/tls/certs/ca-bundle.crt path is RedHat/CentOS/Fedora's convention for the CAfile. You are probably running on Debian or Ubuntu, where the conventional path is /etc/ssl/certs/ca-certificates.crt and the path it's looking for does not exist.
You can work around this by exporting CURL_CA_BUNDLE
so that pysam's libcurl will look for these files in the right place:
export CURL_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt
As for why this has started happening in 0.22.0, it has to do with how wheels are built. The problem is in the system libraries that are distributed inside manylinux wheels, so this is related to #1097 and #1276.
The pysam-0.21.0 Linux wheels were built for manylinux_2_24
, which is now EOL and was based on Debian 9. These wheels contain a copy of libcurl-9f97daa0.so.4.4.0, which as it was built on Debian defaults to the /etc/ssl/… path.
More recent pysam releases' Linux wheels have been built for manylinux_2_28
, which is based on AlmaLinux 8. These wheels contain a copy of libcurl-14d1b62d.so.4.5.0, which as it was built on a Red Hat-compatible defaults to the /etc/pki/… path.
Fedora (40, at least) contains symlinks under /etc so that both styles of path point to the same certificate bundle, so works with both flavours of wheel. (But e.g. Rocky and Alma do not; see also this bug.) Debian and Ubuntu do not, so only the wheel containing the Debian-style libcurl.so will work (without assistance from the environment variable).
This would appear to be a limitation in manylinux's claim to be making wheels that are portable across distributions!
This can be worked around by having everyone set $CURL_CA_BUNDLE
as appropriate, but that is less than ideal. Ways of dealing with this when building future wheels would include:
manylinux_2_24
is EOL, reverting to building that flavour of wheel is a non-starter.CURLOPT_CAINFO
accordingly, so that it would automatically work with whichever path style was present.The real problem here is the large number of system libraries that get pulled into our manylinux wheels.
If we omitted the plugins from pysam wheels, libcurl.so and many other libraries would not be pulled into our wheels, and this and the two issues mentioned above would be fixed at a stroke. These plugins should not really be shipped within the Python world at all; it would be better if pysam could access externally-provided (non-Pythonised) plugin object files, via $HTS_PATH
if necessary. But transitioning to that model would be a non-trivial deployment problem.
Long-term the correct approach is (4) as it solves numerous problems: these three issues and also reduces the size of our wheels. It may be worth doing (3) too in the interim.
From my POV, the fix suggested works for us! Thank you so much, and if you all feel that the root issue should be carried on elsewhere instead of in here, feel free to close.
Glad to hear it does the trick.
Let's keep this one open to represent the interim fix (3), and in due course I'll open another issue to represent (4).
This started happening for us in pysam 0.22.0; it doesn't happen in 0.21.0.
In case it helps, here is pip list and a quick example of the error:
The main error seems to be
But it seems odd that this error doesn't happen with pysam 0.21.0 or lower...