olokelo / jxlpy

Cython bindings and Pillow plugin for JPEG XL
MIT License
45 stars 12 forks source link

"Couldn't recognize the image format for file" on big-endian (PowerPC 32-bit) #3

Open wyatt8740 opened 2 years ago

wyatt8740 commented 2 years ago

I have compiled libjxl (with libhwy built with SIMD instructions disabled) on my Powerbook G4 running Debian. The library works fine on my little-endian systems (x86, ARM) but fails on the big endian one. The standalone cjxl and djxl tools are working on the powerbook, so I know that my library build is functional.

My error (in MComix3, but my own fork that has imports for jxlpy in appropriate locations) is [ImageHandler] ERROR: Could not load pixbuf for page <n>:GLib.Error('Couldn't recognize the image format for file ""', 'gdk-pixbuf-error-quark', 3)`.

My guess (without looking at the code) is that the library is doing a 'magic word' check for some hexadecimal number, which has the bytes in opposite order on big endian hardware.

I think that something like what's being done here may be in order? Not really sure.

olokelo commented 2 years ago

Hi, thanks for opening the issue!

For the last couple of days I've been trying to compile libjxl on PowerPC 32-bit using Debian 10 port running QEMU. Sadly I ran into some really weird compilation errors while building it. May I know how did you succeed with that? So far I wasn't able to reproduce your issue :( Honestly I've never used PowerPC architecture before...

If I were you, I would look in 2 places: where I assume that the image is native endian (this is based on imagecodecs implementation) and in the Pillow accept function. Sometimes Pillow's behavior is weird so that if there's an error trying to load the image, it would say the image isn't recognized.

If there's a problem with magic word checking, can you please look at the hex view of the image you're trying to open and see if it's really different than in the _accept function?

wyatt8740 commented 2 years ago

Yeah, to build libhwy I had to do it with all SIMD support turned off; if I remember correctly you need to tell it to be "scalar" in a header file somewhere. I also had to comment out the lines in the libjxl makefile relating to some submodule of it that depends on node.js somehow or another - since Blink/V8 still doesn't care about ppc32. It's totally unnecessary to make a working jxl library (tested with cjxl and djxl), so I'm not sure why it's burned into the makefile. Maybe it was WASM related.

I'll try to hop on the powerbook sometime soon to get a more concrete description of what I did; I remember I also had to struggle for a long time to get it to work. I'll also try what you said. I need to sleep now, though.

wyatt8740 commented 2 years ago

Hey again, quickly before bed I just ran git diff on my sources on the powerbook. They're for older git revisions (see the contents of the patch files for the hashes).

These worked on my Debian Sid installation. Note that I think highway has been fixed upstream so that it falls back to a scalar build automatically now on ppc32. patches_highway_jpegxl.zip

It looks like in libjxl itself I commented out an "emscripten not found" error, which should not have been fatal to begin with. It seems I also hardcoded the thread count to 1 since my machine is single cored, too, so that I wouldn't have to use --num_threads every time.

Thanks so much for trying this; I know it's an obscure platform now and a lot of effort to go through for it.

olokelo commented 2 years ago

Hi and thanks for patches :)

Lastly I've successfully installed libjxl and jxlpy on my ppc32 Debian. Unfortunately I wasn't able to reproduce your issue. On my machine everything works fine (tested all scripts in examples/ directory and decoded some images with no problems).

Can you please try running examples yourself to see if there's a problem with them and if you can, you may provide me the image which your code is failing with?

Also that's not a problem, I really like exploring different architectures and OSes so I'm happy I had to install a port of Debian on ppc32 :D