cgohlke / imagecodecs

Image transformation, compression, and decompression codecs
https://pypi.org/project/imagecodecs
BSD 3-Clause "New" or "Revised" License
111 stars 21 forks source link

JPEGLS not available under devbox/nix #87

Closed jpambrun closed 7 months ago

jpambrun commented 7 months ago

I am experimenting with devbox which is a helper around Nix. With the below config I get the following with no other output

pip install imagecodecs
python -c'import imagecodecs;print(imagecodecs.JPEGLS.available)'
False

I imagine this is some quirk of Nix, but after several hours of digging I still have no lead..

my devbox.json is as follow:

{
  "include": [
    "plugin:python"
  ],
  "packages": [
    "python39",
    "python39Packages.pip@latest"
  ],
  "shell": {
    "init_hook": ". $VENV_DIR/bin/activate",
  }
}

I have these files suggesting the libs are there:

$VENV_DIR/lib/python3.9/site-packages/imagecodecs.libs/libcharls-69f7967a.so.2.4.2
$VENV_DIR/lib/python3.9/site-packages/imagecodecs/_jpegls.cpython-39-x86_64-linux-gnu.so
cgohlke commented 7 months ago

I don't think I can help much. The wheels for linux were built with cibuildwheel on pypa's manylinux2014_x86_64 docker containers. All tests passed. My understanding is that the manylinux_2_17 wheels should work on any glibc-2.17 or later based mainstream distribution. Maybe devbox uses an older glibc or is not "mainstram"? Do all/any other codecs load?

jpambrun commented 7 months ago

You did help! You pointed me in the right direction. Using ldd -r to find which libc (it was 2.38 from a Nix store) revealed that libstdc++.so.6 was not found. Adding stdenv.cc.cc.lib did solve it.

BTW, most codecs did work without libstdc++. I sampled 10+ and JPEGLS and JPEGXL turned up false.

With respect to imagecodecs, my understand is that it swallow the error in favors of setting JPEGLS.available to False? This makes debugging very challenging. I think this can be improved.

cgohlke commented 7 months ago

So the Linux distribution was not manylinux2014 compatible because it was missing libstdc++.

my understand is that it swallow the error in favors of setting JPEGLS.available to False?

That's by design. Codecs may be missing for many reasons. A delayed ImportError exception is raised if one attempts to use a missing codecs's encode or decode functions.