Can't get kakadu to round-trip

[jcupitt]

kakadusave seems to more or less work now, but it's not round-tripping correctly, and I'm not sure why. Can I ping you on this @scossu ?

I tried with this sample PPM file (just to be certain my input file was OK):

https://filesamples.com/samples/image/ppm/sample_640%C3%97426.ppm

It looks like this as a JPEG:

I tried encoding it like this (about the simplest example in the kakadu docs):

$ ./kdu_compress -i ~/pics/sample_640×426.ppm -o x.jp2 Qfactor=85 Clayers=5
Generated 1 tile-part(s) for a total of 1 tile(s).
Code-stream bytes (excluding any file format) = 66,427 = 1.949149 bits/pel.
Compressed bytes (excludes codestream headers) = 66,016 = 1.937089 bpp.
Body bytes (excludes packet and codestream headers) = 65,077 = 1.909536 bpp.
Layer bit-rates (possibly inexact if tiles are divided across tile-parts):
        0.381162, 0.593955, 0.935035, 1.367165, 1.949149
Layer thresholds:
        43782, 43482, 43182, 43023, 0
Processed using the multi-threaded environment, with
    32 parallel threads of execution

But the image I get is almost unrecognisable decoded with openjpeg. I see:

$ vips copy x.jp2 x.jpg

To make:

I tried decoding with kakadu:

$ ./kdu_expand -i x.jp2 -o x.ppm
Consumed 1 tile-part(s) from a total of 1 tile(s).
Consumed 66,425 codestream bytes (excluding any file format) = 1.949090
bits/pel.
Processed using the multi-threaded environment, with
    32 parallel threads of execution
$ vips copy x.ppm x.jpg

But the result is bad again:

At least kakadu and openjpeg decode mostly agree heh. Did I get the encode command right? Have I perhaps built kakadu incorrectly?

[jcupitt]

The simple_example_c demo application and our kakadusave give approximately the same result.

[jcupitt]

@palemieux, could I ping you on this too? Sorry to ask such a basic question.

[scossu]

I will test this later tonight and get back to you.

[jcupitt]

I tried rebuilding without HTJP2K and has the same result. I tried Clayers=3 -num_threads 1, same result again.

[scossu]

I get the same exact info output and the same botched image with your parameters (Qfactor=85 Clayers=5). Also without parameters I get a different but similarly botched result.

The same happens with a TIFF and a JPEG. I suspect something is wrong with the latest version of kdu_compress at this point. I will try compiling 8.3 and trying again.

[jcupitt]

If there's an error, it's probably deeper in the library, all the kakadu compress programs I've tried make images like this.

[scossu]

Which Kakadu version are you testing with?

[jcupitt]

I'm using v8_3-02172N.zip

[scossu]

I compiled v8_3-02108C and tried again, with the same results. There must be something else because that is the version we have in our production image servers.

[jcupitt]

Could you confirm that the kdu_compress command works with your production servers?

./kdu_compress -i ~/pics/sample_640×426.ppm -o x.jp2 Qfactor=85 Clayers=3

If it does, then I suppose it must be something dumb about the way I've built it.

[scossu]

Your command produces a corrupted image in the Docker container that we are using in our production servers.

The same command on the same image outputs a valid JP2 on Mac OSX (silicon).

On the other hand, the command we are using in production to produce HTJ2K produces a valid HTJ2K:

kdu_compress -i /data/sample1.ppm -o /data/sample1.jp2 -rate 3 Qfactor=90 Cmodes=HT ORGgen_plt=yes Creversible=no ORGtparts=R ORGgen_tlm=9

At this point I'll have to involve the Kakadu developers. But in the meantime, can you test kakadu-vips with the above command (for HTJ2K) until we get this clarified?

[scossu]

(^^ FYI I corrected a typo: it's -rate 3 not -rate 2)

[palemieux]

Using KDU 8.4.1

kdu_compress -i sample_640_426.ppm -o x.jp2 Qfactor=85 Clayers=5 kdu_expand -i x.jp2 -o x.ppm

results in a PPM file that looks right in GIMP.

[scossu]

@palemieux which OS/arch?

[palemieux]

I did a quick VS 2019 compile on Windows 11.

[scossu]

It could be a specific arch issue. 8.4. also works for me on OSX/ARM.

[jcupitt]

I'll try 8.4.1 here.

[palemieux]

Works for me on Ubuntu 20 x64 intel

[scossu]

@palemieux did you use the default options for compiling? And did you use the make/Makefile-Linux-x86-64-gcc or the individual ones in the coresys and managed directories?

It might sound like a stretch but I'm trying to find out what is different between our runtime environments.

[palemieux]

mv srclib_ht/ srclib_noht/
mv altlib_ht_opt/ srclib_ht/
cd make
make CXXFLAGS=-DFBC_ENABLED -f Makefile-Linux -x86-64-gcc all_but_jni
cd ../bin/Linux-x86-64-gcc/
LD_LIBRARY_PATH=../../lib/Linux-x86-64-gcc/ ./kdu_compress -i /mnt/c/Users/pal/Documents/sample_640_426.ppm -o /mnt/c/Users/pal/Documents/x.jp2 Qfactor=85 Clayers=5

[jcupitt]

8.4.1 is working for me (ubuntu 23.10)!

I suppose this is an 8.3 bug, I'll close.

[scossu]

Strange, I thought I saw it in 8.4 too. I'll investigate further. But at least you can go ahead with your testing.

Thanks for checking in, @palemieux .