ente-io / ente

Fully open source, End to End Encrypted alternative to Google Photos and Apple Photos
https://ente.io
GNU Affero General Public License v3.0
16.02k stars 821 forks source link

Support uploading of JPEG XL #595

Closed vishnukvmd closed 8 months ago

vishnukvmd commented 1 year ago

JPEG XL is an efficient format for storing images. However the format is yet to gain adoption: caniuse.com/jpegxl

On the brighter side of things, FFmpeg now supports this codec. Which means we can generate thumbnails and render them on desktop.

There's also an NPM package (jxl-wasm) that provides a CLI for encoding and decoding, which we could reverse engineer. But FFmpeg will be a better bet since we're already using it for transcoding and generating thumbnails for videos.

Rendering these images on mobile will be a challenge until the underlying platforms support them (unless we ship FFmpeg binaries to decode them there as well, which might not happen in the near future). But if there is enough demand, we could allow uploading them from desktop, ignoring the rendering issues on mobile, gracefully handling them with a "This format isn't supported yet." toast.

dhcgn commented 1 year ago

Thank you for creating this issue and considering the use of JPEG XL in this project.

From my point of view, it is particularly these advantages that are interesting:

Key Features The JPEG XL codec is designed to allow users to balance three primary criteria for their application:

  1. high fidelity to the source image (closely matching human perception),
  2. encoding and/or decoding speed,
  3. compression ratio (typically 20:1 to 50:1)

Key features of the JPEG XL codec are:

  1. improved functionality and efficiency compared to traditional image formats (e.g. JPEG, GIF and PNG),
  2. progressive decoding (by resolution and precision),
  3. lossless JPEG transcoding,
  4. support for both photographic and synthetic imagery,
  5. graceful quality degradation across a large range of bitrates,
  6. perceptually optimized reference encoder,
  7. designed for wide gamut and HDR,
  8. support for animated content,
  9. efficient encoding and decoding without requiring specialized hardware.

In terms of compression performance, key results are:

  1. Lossless JPEG transcoding reduces JPEG size by around 16% to 22%.
  2. Lossless compression (from pixels) is around 35% smaller than (optimized) PNG for 8-bit images, and the improvement is even larger for images with a higher bit depth (e.g. 10-bit or 12-bit).
  3. JPEG XL is visually lossless at about half the bitrate required by JPEG, i.e. at compression ratios of around 20:1 (1.2 bpp) where JPEG would compress only around 10:1 (2.4 bpp).
  4. Compared to other image codecs, JPEG XL provides superior compression at a given visual quality.

From White Paper: ISO/IEC JTC 1/SC29/WG1 N100400, JPEG White Paper: ​JPEG XL image coding system v2.0, 2023.

vishnukvmd commented 1 year ago

Looks like things might move forward on Chromium: https://bugs.chromium.org/p/chromium/issues/detail?id=1451807#c16

dhcgn commented 1 year ago

One of the major image processing software Adobe Lightroom now supports JPEGL-XL in these formats:

image