low memory implementation for lossless transformation

[jaisoc]

I have been experimenting with your version of libjpeg-turbo and its works like a charm for low memory devices. And importantly there is no significant hit in performance.

However, for lossless transformation the DCT coefficients still need to be loaded into memory. Do you think you could do something similar to what you already did for progressive JPEG decode.

[komakai]

I haven't looked at JPEG lossless transformation before. If you can get me an image file and a command-line that demonstrates the high-memory usage then I'll take a look and see if there is anything I can do.

[jaisoc]

jpegtran_logs.zip

jpegtran in the library uses lossless transformation. It reads the DCT coefficients of the input image. To transform the image it uses matrix manipulation for which it needs an intermediate buffer (workspace_coef_arrays). After the transformation is applied it will generate an output jpeg file with the new DCT coefficients.

I have attached the logs after rotating the input image by 90 degrees The attachment contains the following files.

1. enable_jpegtran_logs.patch - This will help generate useful logs while executing jpegtran and also highlight the high memory usage while executing jpegtran
2. Tulips.jpg - Input image of size 768x1024
3. output.log is the log message generated by jpegtran from the below command "jpegtran -verbose -rotate 90 -outfile Tulips_90.jpg Tulips.jpg > output.log 2>&1"
4. Tulips_90.jpg - Output Image
5. jpegtran_memory_usage.xlsx - Shows a theoretical calculation of the memory usage in jpegtran.

In my current usecase, I'm not actually interested in generating the output jpeg file. If the jpeg_read_raw_data or jpeg_read_scanlines could return a transformed image that would have been ideal for my low memory device. Since that is not supported currently, I have to generate an intermediate jpeg file that transforms the image and then decode the intermediate jpeg file with jpeg_read_raw_data/jpeg_read_scanlines. This approach is slower and memory intensive too..

From your description of how you have implemented low memory progressive decode, it appears it might be easier for jpeg_read_raw_data/jpeg_read_scanlines to actually output a transformed image. Let me know if you can help.

Thanks

[komakai]

@jaisoc thanks for the explanation. I will take a look and see what can be done. In theory the same approach as the low memory progressive decode (creating a bunch of contexts and then jumping around the file decoding small parts of it at a time and switching back and forth between contexts) should work but I'll need to have a hack around first to check if this really is possible.

[komakai]

@jaisoc So if my understanding is correct you don't actually care about loseless JPEG - you just want to be able to rotate a JPEG and currently jtransform_execute_transformation is the only way you have to do that. If you could do something like:

    jpeg_set_read_transformation(&cinfo, JXTRAN_ROTATE_90); /*set a flag specifying the image should be read-in rotated */
    while (/* have data*/) {
        num_scanlines = jpeg_read_scanlines(&cinfo, dest_mgr->buffer,
 dest_mgr->buffer_height);
        /* do something with the read data */
    }

that would let you do what you want to do - correct? (jpeg_set_read_transformation doesn't exist yet - this is just me thinking what it would look like)

[jaisoc]

@komakai Correct. That will save me an intermediate step of having to create a rotated image too which again need memory.

[jaisoc]

@komakai Were you able to make some progress? Do you think this is doable?

[komakai]

@jaisoc basically it looks doable and it looks like an interesting challenge, but I'm not sure when I'll get round to it. Could be a couple of months before I get some time to work on it.