Closed wayfarer3130 closed 1 year ago
chcunningham
commented
2 years ago Support for higher bit depths is definitely on the road map. Supporting 1 bpp formats should also be doable. Which 1 bpp format(s) are most common?
I'm a little confused about "additional channels"... do mean something more than an alpha channel? Can you point to an example of a codec API that does what you want?
Rendering HDR VideoFrames/Images is somewhat blocked on Canvas support (context here and here).
wayfarer3130
commented
2 years ago By additional channels, I mean things like region identification (segmentation in medical parlance), or additional wavelengths of light (infrared, fluoroscopy etc). The idea is to do things like being able to click on a location and have an overlay that indicates what type of object is at that location, or being able to correct for lighting types in an image, or show what the image would look like under black light. Yes, I know one can add those as extra images, but it would be nice to have it all in one place, as the data actually correlates to the image object. The segmentation might be a single bit - for example "heart" might be a single channel, and there might be a bunch of channels for different things, or it might be an identifier lookup based on, say value 3 int he "identifier" channel.
Bill
On Tue, 18 Jan 2022 at 13:04, chcunningham @.***> wrote:
Support for higher bit depths is definitely on the road map. Supporting 1 bpp formats should also be doable. Which 1 bpp format(s) are most common?
I'm a little confused about "additional channels"... do mean something more than an alpha channel? Can you point to an example of a codec API that does what you want?
Rendering HDR VideoFrames/Images is somewhat blocked on Canvas support (context here https://github.com/WICG/canvas-color-space/blob/main/CanvasColorSpaceProposal.md#additional-color-spaces-high-bit-depth-and-high-dynamic-range and here https://github.com/w3c/ColorWeb-CG).
— Reply to this email directly, view it on GitHub https://github.com/w3c/webcodecs/issues/427#issuecomment-1015675661, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGT56XKXJMNVNZUPK7O4PYDUWWTUVANCNFSM5KOF64TQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID: @.***>
sandersdan
commented
2 years ago I'm familiar with this sort of use in the FITS image format for astronomy, but I'm not aware of any image format supported by browsers with these sorts of features. Are you asking for just the API portion so that a polyfill could implement this, for browsers to support a new format, or is there an existing, supported format that we simply need to expose more features from?
wayfarer3130
commented
2 years ago JPEG XL supports all of those features, and so does JPEG XR. I'm voting on JPEG XL because of the backwards compatibility with JPEG and the higher compression ratios at very similar CPU cost to decompress. It also supports HDR lossless images, which is really what I'm interested in for display in a browser.
The extra channels aren't really required, but if one is going to handle direct access to the HDR data before converting it to 8 bit RGB, it would be really nice to allow access to all the channels/data, and not just grayscale or RGB data. The access for medical images is used to show the detail in, for example bone or in muscle - the ranges for each are so different that showing bone data would make the muscle completely black or white, or if you show both, then you lose so much detail in both of them that neither one is useful.
For something like a game, imagine using lossless high bit depths to store HDR to allow displaying the same image from day to night time. Then, if you also have extra channels, then you can do things like using an extra channel to identify items in the game that can be clicked on, or, to automatically allow highlighting certain objects by using a different channel as an alternate alpha. There are lots of things one can do with additional image bits and channels.
Thanks, Bill
Bill
On Tue, 18 Jan 2022 at 14:23, Dan Sanders @.***> wrote:
I'm familiar with this sort of use in the FITS image format for astronomy, but I'm not aware of any image format supported by browsers with these sorts of features. Are you asking for just the API portion so that a polyfill could implement this, for browsers to support a new format, or is there an existing, supported format that we simply need to expose more features from?
— Reply to this email directly, view it on GitHub https://github.com/w3c/webcodecs/issues/427#issuecomment-1015749440, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGT56XOAAUIKDCW4MOAHV53UWW4ZRANCNFSM5KOF64TQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID: @.***>
dalecurtis
commented
1 year ago Duplicate of #384
Several of the image formats and video formats now defined allow defining both additional channels and higher bit depths (9+). Access to this data as raw data is critical for some types of applications such as:
It would be extremely helpful to have the webcodecs API include access to the raw image data, any number of channels as not just byte data but also 16 or 32 bit data, as well as 1 bit data (because accessing the data as individual 1 bit in 8 bit objects is very inefficient).