Open thelordofcorruption opened 2 years ago
Very maybe. I like some things about the format, but I'd want to figure out if there's a patent problems, and if browsers actually turn Jpeg XL on by default.
Thanks for looking in to it! Something like AVIF (https://github.com/renpy/renpy/issues/2977) might be similar or better in some cases then JPEG XL in lossy formats but AVIF seems to be really bad ( https://github.com/AOMediaCodec/av1-avif/issues/111) when it comes to any lossless compression producing images that are sometimes larger then PNGs and not being true lossless since it needs to convert RGB to YUV so I think JPEG XL would be a good format for both lossless and lossy and worst case scenario it will be better for lossless.
Thanks for looking in to it! Something like AVIF (#2977) might be similar or better in some cases then JPEG XL in lossy formats but AVIF seems to be really bad ( AOMediaCodec/av1-avif#111) when it comes to any lossless compression producing images that are sometimes larger then PNGs and not being true lossless since it needs to convert RGB to YUV so I think JPEG XL would be a good format for both lossless and lossy and worst case scenario it will be better for lossless.
I'm somewhat agree that maybe in loseless JXL is better than AVIF but it has one huge problem - JXL have very slow encoding speed and since it don't have tools like avifenc/avifdec (that can fully load CPU) compiled with very fast rav1e+dav1d codecs that take a few seconds (against about a minute for JXL) to encode/decode AVIF files JXL wouldn't get popular like AVIF.
I don't think these are very relevent reasons for/not to include support for these formats in renpy. For example FLAC is very tedious to manipulate and to edit, yet we support it because it's relatively easy for us to do so. Same for the WEBP format, which Windows doesn't even natively recognize (without bugs). I think the major criterions would be 1) technical support on the various platforms renpy needs to run on (incl. android and web browsers) 2) patent and license issues with the decoder codec 3) speed and performance when decoding the image (not encoding it) and finally 4) creator use of the new formats
Slowness of encoding can be an obstacle for point 4) to happen, but I don't think it will be an obstacle for supporting it in renpy.
I don't think these are very relevent reasons for/not to include support for these formats in renpy. For example FLAC is very tedious to manipulate and to edit, yet we support it because it's relatively easy for us to do so. Same for the WEBP format, which Windows doesn't even natively recognize (without bugs). I think the major criterions would be
- technical support on the various platforms renpy needs to run on (incl. android and web browsers)
- patent and license issues with the decoder codec
- speed and performance when decoding the image (not encoding it) and finally
- creator use of the new formats
Slowness of encoding can be an obstacle for point 4) to happen, but I don't think it will be an obstacle for supporting it in renpy.
Actually I've never said that JXL should not be in Ren'Py... Btw I've tested it with most recent build (JPEG XL encoder v0.8.0 bb96fb0) and speed was good (both loseless and lossy) and in loseless mode JXL beat AVIF by speed (because in loseless mode AVIF uses slow aom encoder) and by size (actually very big difference).
I don't think these are very relevent reasons for/not to include support for these formats in renpy. For example FLAC is very tedious to manipulate and to edit, yet we support it because it's relatively easy for us to do so. Same for the WEBP format, which Windows doesn't even natively recognize (without bugs). I think the major criterions would be
- technical support on the various platforms renpy needs to run on (incl. android and web browsers)
- patent and license issues with the decoder codec
- speed and performance when decoding the image (not encoding it) and finally
- creator use of the new formats
Slowness of encoding can be an obstacle for point 4) to happen, but I don't think it will be an obstacle for supporting it in renpy.
Actually I've never said that JXL should not be in Ren'Py... Btw I've tested it with most recent build (JPEG XL encoder v0.8.0 bb96fb0) and speed was good (both loseless and lossy) and in loseless mode JXL beat AVIF by speed (because in loseless mode AVIF uses slow aom encoder) and by size (actually very big difference).
Yeah JXL is not slow at all but a lot faster and uses multiple threads when encoding. libjxl is a thing (https://github.com/libjxl/libjxl) but I don't think it's relevant since SDL_image probably handles the decoding of images.
Some benchmarks: https://twitter.com/jonsneyers/status/1346389917816008704
I don't think these are very relevent reasons for/not to include support for these formats in renpy. For example FLAC is very tedious to manipulate and to edit, yet we support it because it's relatively easy for us to do so. Same for the WEBP format, which Windows doesn't even natively recognize (without bugs). I think the major criterions would be
- technical support on the various platforms renpy needs to run on (incl. android and web browsers)
- patent and license issues with the decoder codec
- speed and performance when decoding the image (not encoding it) and finally
- creator use of the new formats
Slowness of encoding can be an obstacle for point 4) to happen, but I don't think it will be an obstacle for supporting it in renpy.
Actually I've never said that JXL should not be in Ren'Py... Btw I've tested it with most recent build (JPEG XL encoder v0.8.0 bb96fb0) and speed was good (both loseless and lossy) and in loseless mode JXL beat AVIF by speed (because in loseless mode AVIF uses slow aom encoder) and by size (actually very big difference).
Decoder is open source and licensed under BSD-3 (https://github.com/libjxl/libjxl) and also as mentioned above in the original post jpeg xl is "free, open source and royalty-free". The patent on libjxl is listed here https://github.com/libjxl/libjxl/blob/main/PATENTS but it seems very permissive.
As for support as I mentioned above, Chrome, Edge and Firefox do support the format but it needs to be enabled in settings and as for Firefox you also need a nightly build of Firefox. As for Android, I've not been able to find much info about jpeg xl support.
Even if jpeg xl isn't enabled right now on browsers and maybe doesn't exist yet on android I think it's still nice to have on PCs and Macs. PC and Mac ports are seperate from android / web so you could always use jpeg xl for PC / Mac and webps for Android / Web but I suspect support for web will be enabled soon.
Some of libjxl's dependencies are brotli
, and highway
. These are vendored in libjxl, so if libjxl is to be added only, then using the vendored dependencies should be fine.
However, if WOFF2 support is to be enabled also in freetype, it may be better for the build process to use own brotli
.
+1 to WOTF Support being enabled in freetype aswell
I'm somewhat leaning towards AVIF over JXL, due to AVIF being in all the browsers, and JXL being pulled from chrome.
Unsure why that should matter here? By all means, if one format is more or less strictly better than the other, there's no point in having both.. but otherwise it doesn't seem so automatic.
There's also the size of the Ren'Py download to think about - which matters a lot for the web version and smaller games. I like JXL quite a bit, but I don't really want to have that download size unless the format gets tool support.
I'm somewhat leaning towards AVIF over JXL, due to AVIF being in all the browsers, and JXL being pulled from chrome.
I think that is a bad idea. For multiple reasons but the most relevant being:
JXL
.AVIF
hardly (I tested them myself) (in comparative visual quality)
AVIF
excels when getting an image still from a video encoded as AVIF
making it excellent for video previews and thumbnailsJXL
excels at everything else (including lossless)JXL
has rivals decoding performance, when tuned for better decoding.
AVIF
implementations are started 6 years old and they have ~5 years of optimizationJXL
is 2 years old with only ~1 year of optimizations.JXL
is still badly optimized. One of the lead devs even has good ideas on how to improve its performance tremendously. I'll find his post sometime. I forgot where he wrote it.JXL
is faster to encode and decode. Also, a WASM decoder for JXL
can be fit into 290KB gzipJXL
JXL
JXL
supports that AVIF
doesn't (or not nearly as well)
AVIF
while being acceptable with JXL
(due to how data is organized)
AVIF
is pretty much just a keyframe of a video.AVIF
creates very bad heavily compressed images. It requires more data than JXL
to do low quality images in a human pleasant wayJXL
supports many more colors than AVIF
I believe JXL
is the next format for the forseeable future. Should last as long as png
lasted so far!
So I ask you, @renpytom. Please reconsider.
If there are concerns about download size, I believe JXL+JXL decoder will have much smaller size than AVIF+AVIF decoder.
But since when downloading, the executable is compressed, so it may not be a big of a difference.
So including both AVIF and JXL decoders is not that much of a problem.
I believe JXL is the next format for the forseeable future. Should last as long as png lasted so far!
What makes you think this, given the experience with, say, JPEG2000 - a format that wasn't supported by browsers, that never really went anywhere?
Compare https://caniuse.com/avif vs https://caniuse.com/jpegxl
I like Jpeg XL, quite a bit, but I have trouble seeing it taking off. (I'll re-evaluate this if it does.)
What makes you think this, given the experience with, say, JPEG2000 - a format that wasn't supported by browsers, that never really went anywhere?
J2k
is in patent hell. Is it patented? Is it not? No one wants to risk. JXL
has none of that.JXL
recompresses even better than J2K
for JPEG and JXL
surpasses PNG for lossless compression
JXL
usually wins.J2K
requires (by comparison and context) a huge amount of computation power to encode and decode a J2K
image. J2k
pretty much needs a GPU for a good performance
JXL
performance already surpasses J2K
's while there's still a LOT of performance improvements already being looked into and others not tagged as such.That's what I know related to J2K
for now.
https://photo.stackexchange.com/a/11714/62857
JPEG2000 never went anywhere because it was far too complex for the few improvements that it added, and then its legal status had always been murky until at least a few years ago.
The fact that JXL is already shipping in every desktop browser should be already a big sign that things are can be different this time.
Regardless anyway, I see now how using the built-in capabilities would be pretty darn neat for renpyweb. But if the alternative decoder is just at most 300KB extra to ship, then I just cannot see how that would be a meaningful difference (especially with one single scene being easily just as big). With this said, again, I don't really have an opinion on the technical merit of any standard over the other (AFAIK it should even be that AVIF has both a lossless and RGB mode, even though they may not be as "first citizens" as the rest).. but that's what I feel like this thread should revolve around. Features and inherent shortcomings, not (for lack of a better term, even though I'm not giving it a negative connotation) politics.
Sure, but that's a feature argument. As far as the popularity argument goes, it's only a promise, that JXL has the potential of booming widely. If it does, we certainly will support it, but until then... (I was answering to Brunoais)
If it does, we certainly will support it, but until then...
I just hope not too many think like that... Otherwise, it will be a large sum waiting for themselves... However, I do understand that stance very well.
If I were to make renpy... I think I'd be thinking about Pillow supporting it.
A couple of ideas, again from a very detached prospective:
There are actually two somewhat exclusive features that only each core has though:
Perhaps there are other nitty-gritty quirks that I have missed (perhaps some peculiar difference in HDR, CMYK or whatnot) but AFAICT these seem the elephants in the room.
I'm back leaning towards including JXL - there's apparently a patent issue with the AVIF container that might be problematic.
Even if I'd like to see JXL
in renpy. My research shows that AVIF
has no patents attached, just like JXL
. I don't think incorrect information should be the decider factor.
There might be a patent problem because AVIF is a format that puts an AV1 bitstream into a HEIF container and HEIF seems to be patented by Nokia. I am not a lawyer but I assume this could probably cause issues.
As for JXL even if it does not get web support, it can still be used on desktop and I assume on android if you deliver a native decoder. In my game I deliver lossless images for desktop and those aren't an option on AVIF since lossless images on AVIF seem to take more space then even pngs.
renpytom also mentioned size as a concern, AVIF's libraries are much larger then JPEGXL's are
Size wouldn't be a problem for AVIF, because desktops don't care about it and browsers already support it out of the box regardless. And anyway, as long as we assume you'd want to support AV1, I think 95% of the weight would be already there. Also, I guess that for as much the lossless features parity on paper JXL may have some "quantitative" advantage there.. and of course any use case should be covered. But is there really a practical reason that "visually lossless" isn't enough?
On the other hand I'm really appalled to learn that HEIF is potentially patent encumbered (it doesn't seem like Nokia has ever tried to assert them, and it would be really fucking nuts if netflix had cornered themselves in this amateurish way, but yet there's no explicit green light).
One benefit for jxl is for ongoing project already using jpg (and not keeping lossless backups, which is foolish but a lot of users are not technical), it offers lossless transcoding, so they could gain possible space savings essentially "for free" without degrading quality.
Also, I do think that supporting AVIF through ffmpeg rather than SDL image, might help keep the binary size down, when AV1 support is added.
Very maybe. I like some things about the format, but I'd want to figure out if there's a patent problems, and if browsers actually turn Jpeg XL on by default.
What does it matter? Its an option, people that would select it can sort out where they want to open it. It literally halves the data needed for lossless imaging and every major photo editor can open it. Not to mention the lossy advantages. It should be added. You have bmp and tiff as options for gods sake, is this 1998
I ran some tests this morning and damn the benefits of JpegXL over Webp are strong, even for desktops alone. The benefits don't come so much from the gain in size compared to webp but more from the actual image quality achieved while keeping the size low.
For years I used using Webp lossless, no less than that because Webp even when set at 99% lossy (the next maximum in-line quality setting after lossless), you can see a loss a deterioration in the vibrant color (especially for the red) and a very tiny bit of smearing in certain areas. It made me stay away from 99% webp lossy compression except for android because the screen is tiny and you couldn't tell the difference.
Here is the test result in size, using xnconvert: original file: 3.1M webp lossless: 1.6M jpegXL lossless: 1.5M webp lossy 99%: 619KB jpegXL lossy 99%: 744KB
Now the biggest game changer and what the size alone can't tell, is that jpegXL lossy 99% is subjectively an identical image to lossless conversion, when 99% webp lossy is a far inferior image with smearing and loss in the color spectrum. Basically, for the people that care for image quality, for the first time, it makes lossy something that you can actually use. And this alone makes a huge difference, the day jpegxl is supported is the day many ren'py games in current development, including mine, will have their size halved because lossy is actually 99.99% identical to the original image in terms of quality so it becomes a no brainer to use it.
EDIT: Didn't run an intensive test but Avif seems to be just as good in terms of quality/size at lossy 99%, any of these two seems to be a big improvement over Webp in this scenario.
Just for the very records (and hoping this wasn't also the reason for everybody else in here to nominate lossless compression)... The great majority of times, noticeable losses in "vibrant colors (especially red)" is just chroma subsampling being enabled.
And it's only WebP (being derived from a lousy video codec) that requires a specialized hackjob of a profile to toggle it. You could probably use 90% quality JPG (yes, the original base one, absolutely nothing new or fancy) without downsampling in 4:4:4 mode, and you'd probably still be talking about "subjectively identical".
I wasn't aware, however color is only one part of the equation, what about the smearing on some of the edges, not sure if that is easily fixable, I could send you an example if you have an email and show you how much an image can deteriorate even when set at 99% lossy using webp versus lossless. Which doesn't happen with Avif or Jpegxl. For jpeg, it's always been a great format in terms of size and quality but afaik it doesn't support transparency which makes it a no-go except if you are making a VN with everything merged into one layer, which I think is pretty rare nowadays.
With some formats, you don't need to get a licence if the tool you are using already has it. Probably a good choice to check first, though. From some short research, it looks like Nokia owns a Java Decoder for decoding HEIF containers, not sure if they actually own the format itself.
I could send you an example if you have an email and show you how much an image can deteriorate even when set at 99% lossy using webp versus lossless.
Yes. I know, I understand and I believe it. It is because they aren't just compressing the image.. uh, "spatially" but they are also straightaway removing 75% of the chrominance information. And webp is a pretty shoehorned standard that can either do "full video-like" mode with every shortcut of the situation enabled or "full fidelity" mode with none at all. I just mentioned JPEG because you wouldn't expect the far older and famously far less efficient codec to do better, and giving that a try would immediately confirm or disprove my suspicions.
Then, yes, transparency is important and only jxl or avif (which supports lossy 444 with the high profile) could solve every problem. But if I'm right, I'm afraid this isn't really concerning (even if this or this then could further disrupt "mathematical losslessness"). But who knows, maybe if you try to encode something fancy like "10-bit HDR full-chroma RGB with alpha" one of them will break.
So, spent most of the weekend trying hard to add Jpeg-XL support to Ren'Py.
I'm now fairly convinced that this isn't going to happen until there's a JXL decoder that is written using standard C. The current libjxl implementation is written in modern C++, and trying to get modern C++ cross-compiled to the 14 or so combinations of operating system and CPU that Ren'Py supports... it just was not happening.
This is made a bit worse by the way we target older versions of many of these platforms, where the sysroots that are present don't support features used by modern C++, and by the way that Ren'Py statically links many of its dependencies to make this possible.
My current thinking is that I'd like to include Jpeg-XL decoding in Ren'Py. I plan to do it when there is a C library that can easily be compiled to the platforms Ren'Py supports at the time.
https://github.com/lifthrasiir/j40
Looks like one candidate for such a library, though it currently seems to be missing enough features at the moment that we couldn't deploy it. If anyone is aware of others, please let me know.
One thing that could make it easier to target multiple platforms is to compile e.g. libjxl to WASM using wasi-sdk, then use wasm2c to convert the WASM to C. From there, it should be easier to use the exports provided to make use of the library.
Using this same method, it could also make it easier to integrate libraries that are written in different languages or have difficult build systems, such as Inochi2D.
C++11 doesn't exactly seem all this fancy in the year of the lord 2023 (like, I think even the PSP and PS2 toolchains have some degree of support for C++20 by now). Anyhow, I guess #3906 goes nowhere then?
I'm suspecting I will add avif, as all the browsers seem to have support, which means that there's unlikely to be a major patent problem. (And big pockets to defend against it.) Plus, I experimentally added avif last night, and it built without problems, which is a good sign.
The issue isn't really C++11 vs C++20, it's more going from no C++ to some.
My current plan with JXL is to wait for non-reference implementations to appear. (That's probably a good idea in any case, as it's a sign that the format is being adopted.)
A bit saddening to see what seems to be a slightly better format in Jxl being robbed away by what looks like to be a sort of lobbying from google... But overall Avif is still a pretty big improvement over Webp and from a non-political pov, it makes sense for Ren'py to support Avif since that means you can actually convert your images in the same format and have them working across all platforms I assume.
Anybody already tried https://github.com/renpy/renpy-build/commit/728013da7d29ca7c6cc93756a91b941d9a4934c6 to build/use avif with current renpy-build? Btw animated avif not planned/supported? It' very efficient (regarding file size) - almost AV1 video alternative until it added too.
A bit saddening to see what seems to be a slightly better format in Jxl being robbed away by what looks like to be a sort of lobbying from google... But overall Avif is still a pretty big improvement over Webp and from a non-political pov, it makes sense for Ren'py to support Avif since that means you can actually convert your images in the same format and have them working across all platforms I assume.
I wouldn't consider this to be a malicious intent, when there are other valid reasons to explain this. Jpeg XL compression yields the smallest file sizes, however, encoding and decoding are the slowest of all formats on every architecture.
https://storage.googleapis.com/avif-comparison/index.html
https://storage.googleapis.com/avif-comparison/decode-timing.html
The use of the AVIF format in Ren'py is a better choice, especially if it's faster and more widely supported. I think it would be confusing if certain Ren'py features only worked with certain file formats, and only on specific browsers.
If the format becomes more widely adopted, things may of course change.
https://storage.googleapis.com/avif-comparison/index.html https://storage.googleapis.com/avif-comparison/decode-timing.html
Google's tests are done wrongly. Not done in equivalent comparison position and using color gamuts that JXL wasn't properly optimized for yet (they focused on human perception vs machine perception). They used the worst jxl can deal with and then compared what AVIF deals acceptably with. Jon Sneyers explains that well.
Yeah, tests were wrong and using a version of Jxl that had problems, and for having tested both encoding speed, AVIF is significantly slower for roughly the same quality at the present time (but hopefully will improve quite a bit over time).
https://storage.googleapis.com/avif-comparison/index.html https://storage.googleapis.com/avif-comparison/decode-timing.html
Google's tests are done wrongly. Not done in equivalent comparison position and using color gamuts that JXL wasn't properly optimized for yet (they focused on human perception vs machine perception). They used the worst jxl can deal with and then compared what AVIF deals acceptably with. Jon Sneyers explains that well.
Sorry but this is false.
The linked test results are from December 12 (updated December 14), using the newest version of Jpeg XL (Version: 0.7-base-43) as outlined in the verification process.
You can check the details of why this is a controversial result, by reading the comments (at the end for the most recent ones): https://bugs.chromium.org/p/chromium/issues/detail?id=1178058
In any case for ren'py use case, encoding speed atm is faster on jpegXL. Though can't say for sure about decoding speed, but should be fast enough for a visual novel in any case I think.
Guys, please. Focus. The real deal here is that nobody gives a damn about performance for a visual novel. And neither native compatibility with native platforms is really critical, since we can ship our libraries anyway. (and if people care so much about "neatness inside the browser" then progressive coding seems the best thing since sliced bread)
Again, the problem with JXL was the difficulty of getting the reference implementation running on the many platforms Ren'Py supports, without massively rearranging the build to have dependencies on C++ runtimes that aren't needed anywhere else.
as an update, there are currently, as far as I know two active jxl decoders, a native rust decoder jxl-oxide is a rust based one that has today hit v0.1.0 and was published, and a java one jxlatte of which I am unsure of it's current status.
jxl-oxide does pass a good amount of the conformance test suite (it doesnt seem to be tested against all of it), and performance at least on my desktop isn't bad.
I have no idea how well these may or may not integrate but if either of these looks like it might work for you, maybe worth keeping an eye on.
could you use wasm2c/RLBox to get the reference implementation of JXL running?
(disclaimer: wasm2c contributor here)
Update on the state of things here, based solely on poking around:
Mozilla's opposition to JXL is that it's a massive pile of new C++, which represents a huge possible attack surface, not something they want to put in a browser without a really strong incentive. However, they said in September that they are open to including JXL support if it's implemented in Rust.
As of September, there's an official jxl-rs project underway, and the author of jxl-oxide appears to be contributing to it. No idea how far along it is, how fast it's moving, or how much is left to do.
For reference: jxl-oxide has 36kloc of Rust; jxl-rs has 13kloc of Rust; libjxl has 109kloc of C++ (plus 24kloc of C headers).
If the blocker here is that Ren'Py's build system can't handle a C++ library, then I seriously doubt including a Rust library will be any easier, so this all might be irrelevant :) But I thought it'd be of interest to JXL fans.
Is there a list of platforms + archs that renpy supports that a decoder would need to easily support?
As mentioned in the last two Tom's comments, the most conceptual problem is relying on non-C libraries. And I might even completely pull out of my ass that somebody might even close an eye if a C++ library popped out that was just a single neat header file.. but alas there is only libjxl which is some legitimately titanic monster.
As he pointed out j40 was absolutely THE shit, but unfortunately it was abandoned before coming out of pre-alpha.
Why is relying on non-C libraries in and of itself in issue? As long as it has good Interoperability, Good support as well as an adequate build system.
I understand that it can, in cases, be hard to achieve that with non-C libraries, but I don't see as long as why those are satisfied that it would be a breaker regardless.
JPEG XL is a new image format made by the "JPEG" committee intended as a replacement for jpeg. It support both lossless and lossy compression while having a smaller size. This would be awesome for people like me who like to use lossless images in their VN for the best quality possible and with JPEG XL we will have lower sizes too.
About JPEG XL:
About it's compression:
And as for it's support all major browsers support it but it's not yet enabled by default. Many image viewers and picture editors already do support viewing JPEG XLs.
JPEG XL has also been added in to SDL_image: https://github.com/libsdl-org/SDL_image/commit/742cd132b18c3612e00f5f7d47bc24dbae2e76f9
Some links: