getmango / Mango

Mango is a self-hosted manga server and web reader
https://getmango.app
MIT License
1.69k stars 120 forks source link

[Feature Request] Add jxl(JPEG XL) support #306

Closed tr7zw closed 2 years ago

tr7zw commented 2 years ago

Is your feature request related to a problem? Please describe. It would be really nice to have support for https://jpeg.org/jpegxl/index.html (https://github.com/libjxl/libjxl) due to it's REALLY good file size.

Describe the solution you'd like Bestcase have support to send out the jxl files to the browser(Browsers support it, but apparently not yet by default without enabling it). Also having an option to serverside on the fly turn them back into .png for clients to show.

Describe a small use-case for this feature request This would decrease the library size multiple times. In my test(default settings), a simple Manga page got compressed from 1.86mb down to 228kb without any quality loss. That's about 8 times smaller! Some more numbers playing with the quality setting: 100: 1.171mb(lossless version of the png) 80: 153kb with just small changes in the sub-pixel noise 50: 87kb(21x smaller!) with just a slight loss in fine details(but no apparent blocky artifacts like jpeg's, on a phone/normal zoom level basically not visible). With on-the-fly server conversions, this process could become fully transparent to the end clients that might not support this feature, and if supported it can heavily save on bandwidth for mobile use-cases, while also turning an entire chapter into the size of one page in png format.

Additional context JPEG XL is in the final stages of standardization and its codestream and file format are frozen., so the fileformat itself is save, but currently the main implementation is only in C++. That works for me since for my tools I can use JNA in Java to use the .ddl/.so files, not 100% sure about Crystals abilities to do that. Worst case it needs to be expanded to be able to use external parsers/decoders(which would be possible with pr 305), or keep this issue in the back of the mind till it becomes possible.

Edit: also looks like tachiyomi added support last year https://github.com/tachiyomiorg/tachiyomi/releases/tag/v0.12.1 https://github.sre.pub/leechy/imgsupport can base as an example to auto fall back to .png in case jxl isn't supported?

tr7zw commented 2 years ago

Ok for sanity check also gave it a test with a zip containing only jxl files and currently it's not detected at all.

hkalexling commented 2 years ago

Supporting JXL itself (sending the files directly to the browser) would be trivial - we just need to add it to the list below. I will include this in the coming release.

https://github.com/hkalexling/Mango/blob/b8ce1cc7f1c8ae9f855064c8831b15192d149a6f/src/library/types.cr#L1-L9

For on-the-fly conversion to other formats we can easily link C libraries in Crystal, but I think I would give this feature a low priority. I don't think JXL is used by many users, and including the additional libraries will increase the file size of the binary we ship.

tr7zw commented 2 years ago

Just sending it would be an awesome start 👍

tr7zw commented 2 years ago

Did a test build with jxl added and everything works perfectly fine 👍 ok, but there are some smaller issues, see below. Edge on PC: start with the jxl flag Firefox on PC: Needs Nightly, enable flag in about:config Chrome on Android: toggle flag in chrome://flags Tachyomi: Works out of the box

Guess I'll now spend a moment creating a small tool to convert my entire lib to jxl.

tr7zw commented 2 years ago

Another small note for the readme: when trying to build mango on Windows, git will convert the linebreaks by default, causing compile errors with config.cr due to the \ new lines. Maybe a small note for Windows users to set git config core.autocrlf false.

tr7zw commented 2 years ago

Ok one issue I noticed, sometimes on first opening of a chapter, it loads all images with width=0, height=0 grafik

After a Control+F5 it's fixed. (just F5 doesn't work) Edit: at the same time the log is spamming "Error while writing data to the client"

And to why I care about JXL: I've now converted a copy of one of my libraries to JXL with a quality setting of 50(note that 0 is not "no quality", you can go really far into the - range), and it turned 57GB into 3.8GB(4.5GB on filesystem), saving mindblowing 54GB(or 94%). The only downside is the loss of finer noise, but personally, I'm more than okay with that for Mangas(https://squoosh.app/ is good to see the visual difference). I'll keep an eye open for anything odd other than the 0-size issue.

hkalexling commented 2 years ago

I updated the wiki to include the core.autocrlf config :)

mango-assistant[bot] commented 2 years ago

Hi there! The feature has been added in v0.27.0. Thanks for the feature request!