JPEG XL support

[ChristianMayer]

Is it possible to use JPEG XL, which is lossless and more modern than PNG.

Probably this helps: https://pypi.org/project/pillow-jxl-plugin/

[jhc13]

Thank you for the suggestion.

Unfortunately, I don't think there is a way to add support for JPEG XL in PySide6 yet. The plugin you mentioned works for Pillow, but PySide6 support is also required to display the images.

[geroldmeisinger]

you could internally convert it and also use opencv which supports more formats I think.

[geroldmeisinger]

https://docs.opencv.org/4.10.0/d4/da8/group__imgcodecs.html#gab32ee19e22660912565f8140d0f675a8

LLM:

from PIL import Image
import cv2
import numpy as np

def open_image(file_path):
    try:
        # Try to open the image with Pillow
        img = Image.open(file_path)
        img.verify()  # Verify that it is, in fact, an image
        img = Image.open(file_path)  # Reopen to reset file pointer
        return img
    except (IOError, SyntaxError) as e:
        print(f"Pillow couldn't open the image: {e}. Trying with OpenCV...")

        # Try to open the image with OpenCV
        img_cv = cv2.imread(file_path, cv2.IMREAD_UNCHANGED)
        if img_cv is None:
            raise ValueError("OpenCV couldn't open the image.")

        # Convert the OpenCV image to a Pillow image
        img_rgb = cv2.cvtColor(img_cv, cv2.COLOR_BGR2RGB)
        img_pil = Image.fromarray(img_rgb)
        return img_pil

# Example usage
file_path = "path_to_your_image_file"
image = open_image(file_path)
image.show()  # Display the image

[geroldmeisinger]

https://imageio.readthedocs.io/en/stable/formats/index.html

        img_io = imageio.imread(file_path)
        if img_io is None:
            raise ValueError("imageio couldn't open the image.")

        # Convert the imageio image to a Pillow image
        img_pil = Image.fromarray(img_io)
        return img_pil

[geroldmeisinger]

LLM:

import sys
from PySide6 import QtCore, QtWidgets, QtGui
import imageio
from PIL.ImageQt import ImageQt

# load image using imageio
img = imageio.imread('path_to_your_image')

# convert to PIL format
pil_image = Image.fromarray(img)

# convert to PySide6 QPixmap
qpixmap = QtGui.QPixmap.fromImage(ImageQt(pil_image))

# create a QIcon from the QPixmap
icon = QtGui.QIcon(qpixmap)

https://pillow.readthedocs.io/en/stable/reference/ImageQt.html The ImageQt module contains support for creating PyQt6 or PySide6 QImage objects from PIL images.

[jhc13]

I don't think it's worth putting in that much effort to support a rarely-used image format. Users will have to convert the images to a more widely-supported format if they want to use them for training, anyway.

[geroldmeisinger]

https://imageio.readthedocs.io/en/stable/formats/formats_by_plugin.html Pillow already supports a wide array of image formats. JPG XL doesn't seem to be supported by imageio though. I only found 2-3 image formats which I can export with gimp and pillow doesn't support (dcm, hdr, im1) but imageio does.

integration seems really easy though:

import imageio.v3 as iio
from PIL.ImageQt import ImageQt
from PIL import Image as PilImage

...
    def load_image(self, image_path: Path):
        img = iio.imread(str(image_path))
        pil_image = PilImage.fromarray(img)
        pixmap = QPixmap.fromImage(ImageQt(pil_image))

[geroldmeisinger]

@ChristianMayer I agree with jhc13 here. you can convert it to a supported format, make the captions, and copy back. but even if we wanted to, we would first need support in any other image backend (pillow, opencv or imageio).

LLM:

JPEG XL (JXL) is a next-generation image format that offers better compression and quality compared to older formats like JPEG. To work with JPEG XL images in Python, you can use the imagecodecs library, which supports a wide range of image codecs, including JPEG XL. Here's how you can install the necessary libraries and load a JPEG XL image in Python: pip install imageio imageio-ffmpeg imagecodecs pillow img = iio.imread(image_path, plugin='imageio_jxl')

let me add: supported in the main branch

[geroldmeisinger]

https://github.com/geroldmeisinger/taggui/tree/imageio if any one wants to try with imageio (used in image viewer only, warnings are probably from thumbnails). no JXL support btw!

[ChristianMayer]

@jhc13 and @geroldmeisinger JPEG XL has a huge advantage: it compresses lossless much better than PNG and thus could replace it for the training workflow - and thus get the disk consumption down. It also supports advanced features that might come in very helpful when txt2img advances, like HDR. Of course the training tool does need to support it as well, but when everybody is waiting for the other nothing will move on.

Actually that's one issue with JXL: all browsers were on a good way to support it - until Google pulled the plug, which created a very big reaction in the community as you can see in the huge history of the tracking bug: https://issues.chromium.org/issues/40168998 Even big companies like Adobe wrote in favor: https://issues.chromium.org/issues/40168998#comment62 It stayed like that for a short while, but now the ecosystem is gaining momentum again. Most notably Apple implemented it in the Safari browser https://www.webkit.org/blog/14203/web-technology-sessions-at-wwdc23/ - resulting in over 20% of the internet capable of using it directly: https://desu-usergeneratedcontent.xyz/g/image/1711/40/1711407701261.png

I fully understand that taggui doesn't want to go the extra mile to support it when the base library doesn't. But you might reconsider once it includes JPEG XL support as well.

[geroldmeisinger]

it compresses lossless much better than PNG

what about webp lossless? or a new SSD? ;) https://geizhals.at/?cat=hdssd&sort=r#productlist in June 2024 we are at 50€/TB

[ChristianMayer]

I can't comment on all the different codecs, probably https://cloudinary.com/blog/how_jpeg_xl_compares_to_other_image_codecs gives some impression.

And as long as the NVMe SSDs don't make the long overdue huge step forward in size I can't buy a new one as I've got only one unused slot left in my notebook - which is already much bigger than what's common today. But it's not only me, think of all the people that need online resources to train their models - they need to cope with slow upload speeds to get their images to the trainer. Every saved byte speeds up the training significantly.

[ChristianMayer]

Oh, and never forget: https://imgs.xkcd.com/comics/standards.png