Yellow-Dog-Man / Resonite-Issues

Issue repository for Resonite.
https://resonite.com
141 stars 2 forks source link

JXL Image Support #1562

Open jonnyawsom3 opened 8 months ago

jonnyawsom3 commented 8 months ago

Is your feature request related to a problem? Please describe.

Jpegxl is an emerging format built on good ol' Jpeg and WebP's lossless mode. In recent months it already has full support from Apple and Adobe, with Microsoft working on native Windows integration (Latest Preview builds allow setting as wallpaper, but not decoding yet), so being able to directly load images in Resonite would help improve compatibility while reducing storage and load/save times once it's more widely used.

Describe the solution you'd like

Loading and possibly saving of JXL files

Describe alternatives you've considered

Converting back to Jpeg or PNG beforehand

Additional Context

As mentioned in #880, FreeImage is lacking support due to how new the format is. However other, more modern, image libraries are adding support such as OpenImageIO.

The reference implementation isn't at 1.0 yet, so adding support can be delayed until then, however it is stable enough to be serving 50 Million JXL files per day without issue. (Image sourced from a developer at Cloudinary) image

Reasons to add support include:

The unique ability of transcoding jpegs. This has a guaranteed 20%+ savings in 0.5 seconds for a 4K photo, while being able to recreate the original file bit-for-bit when needed.

A lossless mode built upon WebP (Resonite's current default). At the fastest setting this achieves 30% smaller than PNG in 0.2 seconds for a 4K photo, or smaller than WebP in 1/4 the time. (5.7MB in 2 seconds vs 6.9MB in 9 seconds)

JXL art for placeholders/backgrounds, usually only a few hundred bytes or less for instant load times.

The best Speed/Size ratio of all current formats https://cloudinary.com/blog/jpeg-xl-and-the-pareto-front. Possibly making it a good format for screenshots/Textures in future. (Blender support is in development)

HDR/High Bit Depth support going all the way to 32bit floats. Already being used in scientific institutions and satellite imagery for this reason.

Progressive Loading, which may not apply to Resonite, but would allow images to load in as little as 5% of the data transferred.

I reiterate

This would require either native implementation using libjxl or changing to OpenImageIO currently. So along with broader support not being finished yet, consider this a long term goal that would have long term effects.

Requesters

jonnyawsom3

Frooxius commented 8 months ago

I feel at this stage there isn't that much pressure to add this format over AVIF if we were to implement the support directly. Adding it through switch to a different library would be a better choice there, as this would provide more benefits, but it's also potentially bigger engineering task.

I've looked at OpenImageIO in the past, but this is a native library, which means we need a good C# wrapper to use it. I haven't seen a good one yet, but I also haven't done much research on this. Are you aware of any good wrappers for this library?

jonnyawsom3 commented 8 months ago

OpenImageIO was just an example, but a very quick search comes up with a Unity C# binding with more limited support https://github.com/needle-mirror/com.unity.bindings.openimageio There is also libvips with C#/.Net bindings already https://www.nuget.org/packages/NetVips And I'm sure a few other options too if I refreshed my memory

Frooxius commented 8 months ago

The Unity bindings is the one I found before with quick search, but we need direct C# bindings, we can't use any Unity bindings.

I'd have to look into the libvips, it seems interesting, but we'd need to evaluate.

jonnyawsom3 commented 8 months ago

As I said, it's a long term goal, but if I find anything then I'll add it here. Thanks for taking a look

Frooxius commented 8 months ago

It is, I'm just stating what we'd need to potentially take action on this.

sveken commented 5 months ago

It also has crazy support for heaps of layers, I was going to wait until windows supported it for making a issue but you bet me too it :D