Extension for WebP for smaller images compared to jpeg

[pjcozzi]

By Google. From https://developers.google.com/speed/webp/:

WebP is a modern image format that provides superior lossless and lossy compression for images on the web. Using WebP, webmasters and web developers can create smaller, richer images that make the web faster.

WebP lossless images are 26% smaller in size compared to PNGs. WebP lossy images are 25-34% smaller than comparable JPEG images at equivalent SSIM quality index.

Note that this is not GPU compression; this is a replacement for JPG-like compression.

The Cesium team's initial experience with WebP is very promising.

Ecosystem

• Decode in Browsers
  • Chrome & Edge
  • Firefox support in progress
  • iOS & Safari - ?
• JS/wasm decoders
  • http://libwebpjs.appspot.com/
  • https://medium.com/@kennethrohde/on-the-fly-webp-decoding-using-wasm-and-a-service-worker-33e519d8c21e
• Decode in desktop / native apps
  • Reference implementation: https://developers.google.com/speed/webp/download
• Encode
  • Reference implementation: https://developers.google.com/speed/webp/download
  • https://sharp.dimens.io/en/stable/api-output/
• Probably others

Questions

• Should this be an EXT or KHR extension?
• How should glTF reference this? Directly? via KTX2?
• In the near- and long-term, how does this play with CTTF GPU compression?

[lexaknyazev]

iOS & Safari

Probably never, given that they support a different "JPEG-replacement" format.

Should this be an EXT or KHR extension?

KHR would likely require bringing IP in question, so I'd say EXT.

How should glTF reference this? Directly? via KTX2?

Directly, like JPEGs. KTX2 is for GPU-ready data.

In the near- and long-term, how does this play with CTTF GPU compression?

Nothing in the near term. Looking forward, we may borrow some techniques from it for the future (post-1.0) CTTF versions.

[MarkCallow]

Agree with @lexaknyazev, KTX2 is for GPU ready data.

If you can tolerate uploading uncompressed images to your GPU, i.e. you have vast amounts of memory, and transmission size is a problem then maybe webp is useful. However it is never going to be fully cross-platform. I think it not worth spending time on. We can already get JPEG-levels of compression with the under-development CTTF technology and there is likely room for further improvement and that is where time should be spent.

[zellski]

I'll add just a little agreement and a little disagreement:

• For situations like the work I do at Facebook, where we generate client-specific delivery versions of assets (or where folks are able and willing to host a fallback JPEG version online), having a formalised way to use WebP is inarguably powerful. Asset size is of overwhelming concern for us.
• For the ecosystem at large, it seems like it could come at some real cost to user trust in the "JPEG of 3D" notion. You just can't build a self-contained GLB using this format that's not inherently fragmenting, right?
• Encouraging GPU-friendly textures seems more important for the vast majority of use cases.

[pjcozzi]

We can already get JPEG-levels of compression with the under-development CTTF technology and there is likely room for further improvement and that is where time should be spent.

@MarkCallow do you think this could approach WebP-level compression? If so, when?

In the meantime, WebP does fit a lot of use cases today - in particular in Cesium we literally deal with gigabytes and terabytes of texture data from aerial and satellite imagery - and WebP is making a measurable improvement in file size and download performance inline with their claims, about 30%.

At the least I would expect a vendor or EXT extension, but part of the question here is if there is enough interest for a KHR extension.

[lexaknyazev]

is enough interest for a KHR_ extension.

This might imply bringing the whole WebP file format spec to the Khronos IP Framework.

[MarkCallow]

@MarkCallow do you think this could approach WebP-level compression? If so, when?

We'll have to consult the oracle, Rich Geldreich. How does webp compare with JPEG?

[MarkCallow]

This might imply bringing the whole WebP file format spec to the Khronos IP Framework.

I'm not sure this is so. Core glTF uses JPEG but doing so didn't bring JPEG under the Khronos IP framework.

[pjcozzi]

is enough interest for a KHR_ extension.

This might imply bringing the whole WebP file format spec to the Khronos IP Framework.

Potentially. Draco is one example to consider, jpeg is another. We could discuss on the call and with Google.

We'll have to consult the oracle, Rich Geldreich. How does webp compare with JPEG?

😀

[lexaknyazev]

Core glTF uses JPEG but doing so didn't bring JPEG under the Khronos IP framework.

I think there's nothing to bring there - JPEG patents have expired already.

[pjcozzi]

For those following WebP here, please chime in on the pull request for EXT_image_webp by @OmarShehata: #1534

[donmccurdy]

Closing this issue – let's continue discussion in https://github.com/KhronosGroup/glTF/pull/1534.