Closed aaronfranke closed 2 years ago
How about System.Windows.Media.Color
, which uses floats?
If we were to introduce a new color type, I would like to consider ways to represent linear RGB, LCHab, LSHuv, etc.
@aaronfranke in general this is called 32-bit color, so would the name Color32
not be a better name? This would be similar to Int32
.
I'd say it's more frequent to see colors referred to by their component size when using them in code.
This might be answered elsewhere but why the System.Numerics
namespace? Not that I have a better suggestion, just more curious to the choice.
@Turnerj Mostly because I don't know where else to put it. System.Numerics
is also where the vector types are, so it seems fitting since they are similar in structure (but not similar in purpose).
I'm not convinced adding a new Color
type is the solution here. Everyone will have their own ideas on the correct shape, layout, fields exposed, etc.
It might, however, be beneficial to expose some of the common algorithms and have them operate over Vector4
. DirectX Math provides several functions like that here: https://docs.microsoft.com/en-us/windows/win32/dxmath/ovw-xnamath-reference-functions-color As do other similar libraries like GLM: https://glm.g-truc.net/0.9.0/api/a00143.html, https://glm.g-truc.net/0.9.0/api/a00144.html, https://glm.g-truc.net/0.9.0/api/a00145.html
They would then also logically fit into the System.Numerics
namespace and find parity with several of the other types here that are designed for use in games/multimedia based applications.
I'd also be interested in what others, like @JimBobSquarePants or @rickbrew have to say, as they both maintain large/popular apps/libraries in this space.
This reminds of the old XKCD comic about competing standards.
There's a reason each of the mentioned libraries have their own color type and it's because their specific needs require them.
There's simply too much to consider in the graphics space to allow the design of a small collection of types to cover those needs and since that space requires expert knowledge to develop specific common algorithms, most developers won't want to write them and will choose, instead, to adopt a complete ecosystem within which to operate.
This requires finely tuned color structs which cater for things like rgb working spaces, bit depth, and packing order.
I have to be honest and say that I cannot see an obvious benefit in adding these types. The major graphics players in the .NET ecosystem all have their own types already and this will not provide unification among them.
While I do agree with @JimBobSquarePants that it won't be a perfect solution for anything terribly graphics intensive and they would all likely roll their own for various reasons, I do think a basic Color
type is worth having.
Sometimes, I just want to refer to or accept a colour in a method without bringing in other dependencies or rolling my own type - I want something simple and generally interoperable.
While System.Drawing.Color
does exist and otherwise would fit this purpose, it has potential legacy baggage and some weird quirks (see https://github.com/dotnet/runtime/issues/30208).
From my point of view, either a new Color
type would work or (preferably) fix some of the underlying quirks of System.Drawing.Color
.
Thanks for the feedback @JimBobSquarePants. It's about what I expected and is generally inline with what I was feelingπ
Just for clarification, however, would you feel the same way about having some "color" functions that operate on Vector4 exposed under the System.Numerics namespace (rather than exposing new types)?
The idea being that each library has their own types, but many of the core operations are standardized (hence the support from both OpenGL and DirectX) and often have vector, GPU, or shader acceleration available. So it would be possible for image libraries to reuse some of these core functions without needing to remove or change their existing types.
https://github.com/dotnet/runtime/issues/32418#issuecomment-595397329 has links to the respective API surfaces exposed and would give an example of the kind of functions that would be considered.
I have no doubt that people will have their own use cases for Color types, and some people may choose to use their own (and it's great that C# allows this). However, I don't think that this means we shouldn't have a general-purpose Color type inside of .NET.
Having this is a great idea because it would be useful for anyone who doesn't want to build their own Color type or import a library.
A Color type in .NET Core would also serve as a common interchange format, so that libraries wouldn't have to depend on each other by referencing each other's color types, or doing something like using four floats, or a Tuple of floats as an interchange format. I had a similar discussion with developers before when I proposed adding (Value)Tuple
operators to vectors, and they said it would just be better for System.Numerics.Vectors
to be always included in .NET Core and then libraries can reference those types and use them for conversion.
I don't think it's possible to have a "general purpose" color class/struct. No matter what you implement, there will always be too much "what about this other use case?" going on.
First, I'd highly recommend studying how Windows Imaging Component implements pixel formats : https://docs.microsoft.com/en-us/windows/win32/wic/-wic-codec-native-pixel-formats
Also, it's useful to be precise and careful with the vocabulary around colors and pixels (and pixel formats). They get conflated a lot, they are related and definitely overlap, but they are also not the exact same thing. However, it is difficult to fully separate the concepts, especially because of how they are colloquially blurred together, so you can't actually get too pedantic. As an example, System.Drawing's Color
is definitely a color and not a pixel. It is not suitable to create a 2D array of them and call it a "bitmap" (just look at its fields).
I think I agree with @JimBobSquarePants on this. I think that the idea of having of a general purpose Color
type is a wonderful utopian idea, and would be useful, but I'm not sure how to achieve that given the incredible complexity in this space. Even though I've been working with computer graphics for over 20 years, I'm still consistently baffled by new things I learn about colors every year.
I think a bottoms-up approach has worked well in the past. That is, start with a graphics library (e.g. GDI+/System.Drawing, Direct2D, ...) and define -- within the library itself -- the pixel formats that it can natively work with. I don't think the top-down approach being proposed here is the right place to start.
What isn't working well right now is that every graphics library has all its own definitions of these pixel formats, and there's no standardized way to map between them, how to map to vector/GPU types and back, or even how to map to simpler definitions that are also useful (e.g. enum Colors { Blue, Red, White, Black, ... }
) etc. WIC solves some of this on its own with the concept of format converters, as well as color spaces and color transforms aka color management (https://docs.microsoft.com/en-us/windows/win32/wic/-wic-colormanagement). It works pretty well, but as you can see it gets complicated fast! (and, WIC is extremely unapologetic with its complexity) Also, WIC handles "bulk" conversions (full rows or whole bitmaps). It's still useful to have an efficient way to convert 1 pixel/color at a time, which is easier to achieve in .NET or C++ than it is in COM. How do you map a 32-bit-per-pixel BGRA sRGB color over to 128-bit-per-pixel RGBA HDR? Β―\(γ)/Β― I dunno, but having that functionality would be awesome.
I disagree that the number suffix usually denotes the bits per component. In my code base it represents the total size of the color, e.g. ColorBgra32
is a 32-bit color with B, G, R, and A integer components that are 8-bits each. ColorRgba128F
is 128-bit color with R, G, B, and A floating point components (same as what Direct2D uses). Unfortunately the color space is unspecified, but sRGB is the de facto standard right now. Also, if the number denotes bits-per-component, how do you disambiguate between colors that have the same bits-per-component but a different number of components? In Paint.NET, I have e.g. ColorBgra32
and ColorAlpha8
. Both have 8-bits-per-component, but 4 or 1 components (respectively). The latter is used for alpha masking, and is a very important type for graphics work. As you move towards lower-level graphics libraries like Direct3D, just saying Color8
would be waaaaaaaay too ambiguous.
I think this space will all only get more complicated as time goes on (see also: HDR, good luck understanding that space! π), so I don't think there's any safety in finishing a project like this and thinking it would be the be-all-end-all result. It would need revision every few years, if not more often.
I think there's definitely value in having standardized Color types that map to Vector and GPU types. I'd love if there was some kind of implicit reinterpret_cast
-style ability to do custom mappings here.
In Paint.NET, I've settled on a collection of Color____
primitives that mirror and map to those that are provided by Direct2D and Windows Imaging Component (WIC). Each struct then implements IPixelInfo
so that it can describe itself and be used in generic and format-agnostic code. This is analogous to WIC's IWICPixelFormatInfo
interface:
public interface IPixelInfo
{
int BitsPerPixel { get; }
PixelFormat PixelFormat { get; } // directly maps to WICPixelFormatGUID
}
// "natural" may not be the best term, but it just means the size is measured in bytes, not bits.
// in other words, simple pointer arithmetic can be used for all inter-pixel navigation
// so this doesn't cover 4-bits-per-pixel or some other esoteric formats that are 10 or 30 bits per pixel or whatever (if those even exist)
public interface INaturalPixelInfo : IPixelInfo
{
int BytesPerPixel { get; }
}
// makes a lot of generic coding a lot easier
// the IUnsafeElementAccessor thing is largely unimportant now since the release of C# 7.x and System.Runtime.CompilerServices.Unsafe.*
public interface INaturalPixelInfo<TPixel> : INaturalPixelInfo, IUnsafeElementAccessor<TPixel>
where TPixel : struct, INaturalPixelInfo<TPixel>
{
}
public struct ColorBgra32 : INaturalPixelInfo<ColorBgra32> { public byte B, G, R, A; ... }
Sometimes, I just want to refer to or accept a colour in a method without bringing in other dependencies or rolling my own type - I want something simple and generally interoperable. @turnerj
tl;dr from my novel above, "simple" and "generally interoperable" are extreme oxymorons in this space π
A Color type in .NET Core would also serve as a common interchange format... @aaronfranke
I'm not convinced a "common interchange format" can be achieved in this space. It can certainly be done for a very narrow subset of functionality and use cases, at which point it's definitely not general.
Of course we can discuss what "simple" or "general-purpose" or "generally interoperable" mean, that's why this discussion exists, but I disagree with the notion of "I don't think it's possible to have a "general purpose" color class/struct", the message I'm getting from this is basically 'this is so incredibly complicated that we shouldn't even try' as if there aren't advantages to having something roughly general purpose in .NET.
How do you map a 32-bit-per-pixel BGRA sRGB color over to 128-bit-per-pixel RGBA HDR?
That's the purpose of the byte
constructor, you provide it with a byte
for each channel and the constructor maps it such that the range [0, 255] becomes [0, 1]. If you are expecting some other range, you can modify the resulting Color
as needed (or build your own method).
I'm not opposed to renaming Color8
(or really, even dropping it, I mostly care about the 32 bits per channel floating-point Color
myself since it is the most flexible format).
It can certainly be done for a very narrow subset of functionality and use cases, at which point it's definitely not general.
We can just add more functionality if we determine that a given use case is common enough to warrant such functionality. I'm not saying we will make something and never touch it again, calling it done and good for general use cases, we figure out general use cases by user demand and we always have the option of adding more functionality later.
I laid out most of my concerns in https://github.com/dotnet/runtime/issues/14825#issuecomment-497825808, but I'll add a few replies to the comments here...
@rickbrew It's good to see someone else aboard the WIC train. WIC has a hell of a learning curve, but the more time I spend with it, the more I like the way it handles extensibility. The WIC pixel format definitions are a great example of what it takes to define those in a truly agnostic way.
@aaronfranke If you take away the methods that rely on a specific interpretation of the color data so that your 128bit-per-pixel format is truly generic, how does it differ from (or improve on) Vector4
? Vector4
's X
, Y
, Z
, and W
properties can be easily interpreted as B, G, R, A or A, R, G, B or C, M, Y, K or whatever other format you like. Vector4
also comes with the benefit of being recognized as a JIT intrinsic so it can be passed in a SIMD register automagically and has lots of useful math already defined.
@Turnerj have you looked at System.Windows.Media.Color
? It solves a lot of the design problems with System.Drawing.Color
if you need something to represent an actual color rather than a pixel value.
@tannergooding The DirectXMath color utililty functions could be pretty useful if they were built as extensions that operated on Vector4
. I fear, however, that they will become increasingly irrelevant over time because it's no longer safe to assume that all colors live in either the Rec.709 or sRGB color spaces (or the linear space with the same whitepoint and primaries). Wider color gamuts and higher dynamic range will become the norm over the next few years, so by the time those additions would be useable by any number of developers, they'd be on their way to obsolescence.
they'd be on their way to obsolescence
I see your point to an extent, but that is also why those libraries continue adding new functions each release (for example XMColorRGBToYUV_HD
and XMColorYUVToRGB_HD
are relatively new). The goal of DirectX Math and GLM is to provide the most core functions needed/used by games and multimedia based applications (and so they cover the core standards like ITU-R BT.709
, ITU-R BT.601/CCIR 601
, CIE XYZ
, and IEC 61966-2-1:1999
).
However, the functions are also well-defined and MIT licensed (at least for DirectX Math) and are fairly trivial to implement using the existing Vector4
surface area (and the new functions which allow converting Vector4
to/from Vector128<float>
), so it also isn't something that is "critical" or that would block anyone from providing their own library for these things if needed/desired.
Yeah, I'll walk back from "it's not possible", but I still think it's ultra important to be very specific here. What you want is an ultra specific definition that is capable of being employed casually by folks who don't need or want to dive in and read a 500 page book called "Colors and Pixels: Volume I" π
I think starting with what WIC and Paint.NET do would be a good place to start. The primitive/low-level "types" here should be pixel formats, color spaces, and conversions between them. Any color/pixel struct
would implement interfaces that provide information about the pixel format, memory layout, color space, etc. This strategy permits precision and also leaves a lot of room for expansion later.
Once you have a bunch of structs you can define implicit and explicit casting operators, as well as ways for converting between them. For instance, ColorBgra32
should be implicitly castable to ColorRgba64
*, but going the other way would require an explicit cast. Converting to something like (for example) ColorHsv96F
(HSV, 32-bits per component floating point) would need a converter method or class, especially since you'd be losing the alpha channel (this would technically be a blending operation done "over" an explicit, or perhaps an implicit default, background color), and also be subject to rounding/clipping (it's not always a reversible conversion).
@saucecontrol re: S.W.M.Color
, I think this is similar to Direct2D's color struct with some WIC mixed in. WIC, WPF, and Direct2D all share a common heritage and have a lot of overlap. D2D's color struct doesn't include the color context information, but it is still mostly used by itself rather than packed tigntly together into a bitmap. For example, you use a single color value to instantiate a ID2D1SolidColorBrush
, or for gradient stops, etc.
@saucecontrol 's notes on how WPF's color struct includes the color space is an important one. To fully define a "color" you need to specify the pixel values, the pixel format, the color space, etc. To display the color correctly you need to match it up with the monitor's color profile. An important thing to note here is that storing this extra information is in-line with the goal of having a color/pixel struct that is "general purpose", but in conflict with having one that maps to vector/GPU types. Unless you make that information part of the type system like I do in Paint.NET (see my comment above), at which point you need one struct per variation.
* footnote here ... on Windows the de-facto 32-bit color is BGRA, while 64-bit is RGBA. For some reason I'm not currently privy to. Whatever :)
I think there's definitely value in having standardized Color types that map to Vector and GPU types. I'd love if there was some kind of implicit reinterpret_cast-style ability to do custom mappings here.
@rickbrew, we currently have ref TTo System.Runtime.CompilerServices.Unsafe.As<TFrom, TTo>(ref TFrom value)
and you can do pointer casts as well. So *(Vector4*)color
and *(Vector128<float>*)color
are both valid, assuming color
is 128-bits in length.
The codegen, however, isn't always perfect today and is something we are tracking more generally (we do have zero cost conversion between Vector4
and Vector128<float>
for .NET 5, however).
If you have a more explicit ask or need, feel free to open an issue and we can start the discussion on it π
those libraries continue adding new functions each release
Ah, that makes sense. I was kind of impressed they covered Rec.709 in addition to the usual sRGB, but then they're missing Rec.2020 and Display P3 which are definitely bordering on mainstream at this point.
I could certainly see a demand for a common library of accelerated conversions between those standards-derived formats. I'm not sure the BCL is the place for them, but that would be a good library to have.
@tannergooding Yeah that's something I don't actually have any specific needs for at this time. I've not yet converted over to .NET Core, and I've so far been doing vectorization in native code. At some point I'll jump into Vector4
:)
Sometimes, I just want to refer to or accept a colour in a method without bringing in other dependencies or rolling my own type - I want something simple and generally interoperable. @Turnerj
tl;dr from my novel above, "simple" and "generally interoperable" are extreme oxymorons in this space π @rickbrew
From my perspective of "simple", I'm just wanting the lowest common denominator. Like you've described, there are so many details when it comes to colour that a general Color
type that could do everything is a utopian future. I just want something to pass around without inventing my own types or taking on dependencies.
This is likely based on my view of .NET: I see the runtime bringing all the basic types and functionality to the table that people might reasonably need. These effectively act as a lowest common denominator - never meant to do everything but do just enough in a lot of cases without a developer needing to re-invent the wheel.
When you get to the "I need to handle HDR colours" (etc) point, that's when you start bringing in specialised libraries where people have built that functionality. Kinda like if I wanted to do image processing, I could do some stuff with System.Drawing
but if I'm too restricted by it or have some other requirement it can't fulfill, I'd bring in ImageSharp.
@Turnerj have you looked at System.Windows.Media.Color? It solves a lot of the design problems with System.Drawing.Color if you need something to represent an actual color rather than a pixel value. @saucecontrol
My experience is fairly limited with System.Windows.Media.Color
and while I do know it has .NET Core 3 support, isn't it somewhat intrinsically tied to WPF? I don't know what the future looks like for WPF with .NET on Linux - I mean, I might still be able to access the type but if WPF is a Windows-only technology, I could imagine very few API surfaces by third parties taking in that type.
My experience is fairly limited with
System.Windows.Media.Color
and while I do know it has .NET Core 3 support, isn't it somewhat intrinsically tied to WPF?
I agree, I don't think you can standardize on System.Windows.Media.Color
specifically, for the dependency and platform reasons you give. It's in PresentationCore
which is tied to Win32. But it's probably a good starting point of functionality for steering this proposal.
That struct is pretty heavy weight, containing a ColorContext
(reference type), bool
, float[]
, and both a MILColor
(4x byte
) and MILColorF
(4x float
) structs. So, maybe around 180 bytes on its own plus whatever that float[]
takes up (seems to not always be used). Definitely not usable for high performance, but should be fine in many other contexts.
Conceptually, S.W.M.Color
looks like a discriminated union between a 32-bit sRGB BGRA (8-bits-per-component) and a 128-bit scRGB BGRA (32-bits-per-component, floating point) color value.
I don't know what the future looks like for WPF with .NET on Linux
My guess is, don't hold your breath. One of the blockers for porting WPF anywhere other than Win32/x86/x64 is the C++/CLI compiler. It's stuck on Win32/x86/x64, and I have no idea when or if we'll see arm, arm64, or non-Win32 support there. It's a blocker for Paint.NET as well, I have a lot of native glue code in C++/CLI.
WindowsBase may have a chance of being ported elsewhere, I'm not sure what Win32 dependencies it has. But that just gives you, what, the data binding engine? Nothing useful for directly building a UI.
Yes, x-plat is a definite sticking point with S.W.M.Color
. But it does serve as a reasonable model for what a generic color-defining struct would be. Its other major downside is that it only supports RGB(A) colors, which means you'd need something different to work with CMYK, LAB, YUV, or any other color model. Those could both be solved by defining yet another color type or types, but that gets out of hand quickly if we're talking about a lowest common denominator type.
Adding a third incomplete RGBA-only color definition to augment the two existing ones seems like a mistake. Maybe it's possible to break the dependencies on Windows to create a WPF equivalent of System.Drawing.Primitives
that are managed-only and cross platform. That would still have shortcomings, but at least it wouldn't pollute the API any further.
And as @rickbrew pointed out, that concern is orthogonal to the need for something that represents pixel values that you would store or process in bulk, because it would be far too heavy.
For at least 20 years, I've heard developers saying "I just want to be able to say 'Red' and have it mean 'Red'". The reason that problem hasn't been solved in all that time is because it literally can't be. The closest we ever came was sRGB and the W3C's adoption of sRGB as the one true color space. That's about to go back out the window as display technology has passed the limits of sRGB. How do you express the Red that's the reddest Red my display can show, when Red is already defined as 255,0,0? (See the first image @ https://webkit.org/blog-files/color-gamut/comparison.html if you're using a better-than-sRGB display to see what I mean)
when Red is already defined as 255,0,0?
Semantically speaking, with floating point color math, 1
means 100% intensity, so 1,0,0
will always mean fully red. Floats allow you to perform whatever operations you wish, including representing values outside of the normal 0-1 range, but at some point you still have to convert to bits on a display, at which point the only sensible option is to scale down to the 0-1 range, where 1 is full intensity.
So, on a 10 bits per channel display, while the maximum integer intensity would be 1023
, and therefore 255
would not be the brightest anymore, the maximum float intensity is still 1
.
You're talking about operating on a different scale, not working in a different color space. What if Red in my color space is actually 0.8,0,0.05
instead of 1,0,0
? There's no way to convert without a color profile (or as WIC and System.Windows.Media call it, a ColorContext
).
For a more extreme example, open this image in MSPaint or any other non-color-managed application:
And if you want to blow your mind a few more times, this series of educational articles about color management from 13 years ago is great: http://regex.info/blog/photo-tech/color-spaces-page0
Reading that is what convinced me to remove automatic conversion to sRGB when opening images in Paint.NET, something I almost shipped in v4.2 (July 2019). It's just way too complicated and I would've made things worse.
Yum, bluesberries
Semantically speaking, with floating point color math,
1
means 100% intensity
100% intensity relative to what though? The camera's sensors, my monitor's pixels, my eyeballs?
There's no such thing as 100% brightness, you can always add more photons/energy to light
educational articles about color management from 13 years ago
Ha, that's the exact series that led me to create some of my own wacky profiles for testing.
And yeah, since the spoiler is there, the raspberries (or bluesberries) are encoded in that image as ~ 55,70,170, which how I define Red in my wacky colorspace.
Of course, that's not terribly realistic, but "more red than the reddest red you can imagine" does not simply mean pumping up the red channel from sRGB to greater than 100%
Defining a generic, universal type to represent colors in a way that is agnostic from all perspectives, and at the same time capable for meaningful operations on instances in a convenient and performant manner is pretty much impossible.
In ImageSharp we have overcome this issue by defining a Color
type that keeps it's state completely private.
This Color
type is pretty much useless for anything else than passing color information from A to B in an agnostic way. When some algorithm actually needs to extract and use that information, it should convert Color
instances to pixel instances of a specific representation.
Any other solution would - by design - suffer from the issues already well explained by @rickbrew's comments.
@tannergooding @antonfirsov @rickbrew @aaronfranke et al
Just wanted to chime in with what I had going on a while ago FWIW.
I originally started with something I copied from j-codec @ https://github.com/juliusfriedman/net7mma_core/blob/master/Codecs/Image/ColorSpace.cs
That got rather limited IMHO and I found out was based on libav to some extent which is somewhat surprising... so I dug a little deeper and found out that libav also had some issues with certain things e.g. very weird non real color formats that I could imagine but not see how to get to work easily in libav. (mostly custom alpha formats I think)
I ended up with:
ImageFormat: which is still more wordy than I would like but seems to cover ANY format I could think of.
You have to define your transformation ONLY on the source format as the intermediate is RGB right now but that can changed in the implementation of the Transformations if needed as required by the implementation.
The top level classes are: Media Format Which has the MediaType, Size, Media Components [Which are basically the channels of data because this is supposed to work for Audio and Video in the same way] and finally it's MediaBuffer which is your Memory.
You can then get to Audio, Image or Video specific things:
https://github.com/juliusfriedman/net7mma_core/tree/master/Codecs/Image https://github.com/juliusfriedman/net7mma_core/tree/master/Codecs/Audio https://github.com/juliusfriedman/net7mma_core/tree/master/Codecs/Video
MPEG is also implemented to some degree (MPS, TS PES, Etc)
Hopefully it can be useful if nothing else just to serve as a reference from a different point of view where as in this library all colors are thought of as parts of media component which defines how to turn it into either RGB, HSL, or any other color space required by the application with little effort.
Ideally applications should stay in their own color spaces and not need to transform and if they do then typically that is where acceleration is required to reduce the jitter in the encoding / decoding (transcoding) process and all of the other fun complexities as partially shown above.
I think having something like Color is obvious but then to have it's components and be able to drill down into the raw data is very important. I think we should also consider audio and image as just types of media and let the user decide how he wants to view it (think for a [color]blind or deaf person or otherwise disabled person, they are going to require special trans-coding anyway)
E.g. the input and output could be anything you just have to be able to define them, which is easy using the MediaComponents which tell you what they are for and how big they are (each component and how many there are [how to handle odd components etc])
i.e. I am sure we all know you could literally take audio and make it an image in a certain bit depth and color space and you would definitely see something (potentially the environment or device from where the audio was sampled if you performed the right inverse quantization and filtering but more likely what is known as noise when unrefined.)
I think in my implementation in the end a Generic class would be able to be provided for any Transformation which would be supplied with an optional ColorSpace argument to do the underlying conversion given the information it knows about the size of the components and how most efficiently to ready them.
The problem as stated above is that then you need to transform the Red in one space to the Red in another space and that is what the Transformation does so as long as someone has a point of view on a Red it can be converted to another space but might not look the same depending on that space and the values it supports which is where trans-coding is required. (Transformation)
By the way, even if there is no color type implemented for .NET Core, we should still deprecate System.Drawing.Color
.
What would be the reason for deprecating System.Drawing.Color? It is still used for its intended purpose, with System.Drawing; which itself is based on GDI+ and is actively supported on both Unix and Windows.
There's no point to deprecate it without a replacement. It makes sense to update/improve it, but not just leaving behind.
It's used extensively in game engines for basic colors where 8 bits of precision per channel is acceptable such as console messages. For something advanced, more precision is required for sure. For instance, I've implemented a custom LinearColor structure for the game framework for shader materials and so on. It would be nice to have something similar out of the box in .NET.
Then why not replace it with the proposed Color8
type, and add the proposed Color
type? They work for the existing use cases of System.Drawing.Color
, and are flexible enough to be used for more use cases.
The argument against this is basically that doesn't work for every single use case, but it doesn't need to. We don't need to support many color spaces and such, we can just treat 1.0f
in the proposed Color
as 255
in System.Drawing.Color
.
To me it doesn't make sense to argue against a .NET Color
type because it doesn't cover all use cases, and then also keep System.Drawing.Color
which covers even fewer use cases.
Why not use something like I have proposed , it is extensible enough that I could also add the same format as @nxrighthere posted and would that be sufficient? It will only benefit users of shaders
on GPUS
so I suspect not...
My implementation supports audio and video in the same framework and is able to be SIMD optimized, it's basically a modernized take on what libav did.
I am not saying @JimBobSquarePants's library is bad I just don't think it will ever support Audio easily without a bit of work to add that underlying intermedia
which would be able to marshal the types (if safety was even a concern at this point)...
There are also proposals for sound / media and there previously was also sound support in full framework albeit limited, it would work well enough to achieve what you needed without interop most of the time.
Why just not have the users vote on their favorite graphic / media API and then from there just add a way to get the raw buffer if not already exposed and then we can do that, it's what you guys basically did for libuv right?
Heck for all that we might as well just ask libav to be able to link against them and offer a build option with MEDIA_SUPPORT=....
to the build process.
@aaronfranke, there are lots of people using GDI plus and those users will not benefit from optimizations in the Color
code no matter how advanced because they call into GDI which needs things in a specific way in order to do it's job. Changing Color on them will do no good.
I saw we deprecate all of system.drawing
and system.media
and come up with a complete replacement or fix it.
What we already have covers the basic use-case, which is accessing and interpreting colors in a fairly typical format (8-bits per pixel, 3 color channels + alpha channel). We also have System.Numerics.Vector4
which can be the basis of a hardware accelerated floating-point format (32-bits per pixel, 3 color channels + alpha channel) and covers a good number of additional scenarios, including conversion between formats.
Anything additional starts getting into very specialized territory that likely isn't a good fit for the actual core library and that is better served by more specialized libraries, like ImageSharp, Windows Imaging Component, or others available projects. There is some additional functionality that could be exposed for these basic types we already have that would make them feel "more complete", even for the somewhat limited scenarios they support (basic image manipulation and hardware accelerated graphics).
At the end of the day, what we ship in box will never include stuff to satisfy everybody and dev's will need to create or depend on additional third party dependencies in some scenarios. There are benefits and drawbacks to this, just like there are benefits and drawbacks to tying your audio library and video library together, etc. Having smaller more actionable proposals to extend the existing stuff we expose is likely more actionable and while we can have larger discussions about more complete replacements, that is also a much bigger ask and has to be weighed against existing libraries that are likely doing it better (and likely will always do it better).
What we already have covers the basic use-case, which is accessing and interpreting colors in a fairly typical format (8-bits per pixel, 3 color channels + alpha channel). We also have
System.Numerics.Vector4
which can be the basis of a hardware accelerated floating-point format (32-bits per pixel, 3 color channels + alpha channel) and covers a good number of additional scenarios, including conversion between formats.Anything additional starts getting into very specialized territory that likely isn't a good fit for the actual core library and that is better served by more specialized libraries, like ImageSharp, Windows Imaging Component, or others available projects. There is some additional functionality that could be exposed for these basic types we already have that would make them feel "more complete", even for the somewhat limited scenarios they support (basic image manipulation and hardware accelerated graphics).
At the end of the day, what we ship in box will never include stuff to satisfy everybody and dev's will need to create or depend on additional third party dependencies in some scenarios. There are benefits and drawbacks to this, just like there are benefits and drawbacks to tying your audio library and video library together, etc. Having smaller more actionable proposals to extend the existing stuff we expose is likely more actionable and while we can have larger discussions about more complete replacements, that is also a much bigger ask and has to be weighed against existing libraries that are likely doing it better (and likely will always do it better).
I 100% agree but look @ JCode's HomePage Github
That is why I started my library to enhance what issues they had at the time. It looks like they have also moved onto hardware acceleration where supported, which unfortunately I didn't get all the way to; I did get through Manged Intrinsic's though (just barely)
But I do again agree 100% (DRY and DNRTW)
That typed, it's en entire foundation outside of java... We probably need a .net Multimedia Foundation not to mention call indirect or abstraction over the PCI Bus using Pipes...
We are not that far away with Socket.IOControl honestly.... We could File.IOControl ( think there is already a proposal ) ...
We need that way before we have Color otherwise it does no good (unless your on Temple or Zenith OS)
@tannergooding System.Drawing.Color
uses 8 bits per channel, not 8 bits per pixel, and System.Numerics.Vector4
would give 32 bits per channel, not 32 bits per pixel.
Sorry, simple typo was thinking one thing and wrote another
It also limits you to 4 channels, in the purest form without another class to break out the components. my implementation makes no such limitation right now (only up to 255 channels because of byte but easily changed to int though).
There needs to be a structure of something right in front of the Vector and Behind the Color hence why I suggest Components and Buffers and a BitReading API after you have that it's just how many ops you need to do and in what order.
Otherwise we are just back as Unsafe.As and may as well just use Span to represent everything π¦
Since Color8
will be by far the most used color type, and HDR secondary (based on that the old 8bit color format has been sufficient for most things), I'd prefer this one is just called Color
and the 16bit one would be renamed Color16
instead for the more special-case scenario.
I'm also a little confused why you're adding a second 4x8 color type. Why not just add the extra methods to System.Drawing.Color
, and create a new System.Drawing.Color16
?
@dotMorten The proposed HDR color type is made of four floats, which ideally would be 32-bit single-precision floats (for 23 bits of mantissa) so that it can handle 16-bit HDR and more (23 >= 16), not 16-bit half-precision floats (which would give 10 bits of mantissa, and would not always work correctly for 12-bit and 16-bit HDR). Additionally, 16-bit integers would give high color depth, but would not be suitable for all types of HDR because it can't handle values above 100% brightness.
The idea of calling the float version Color
and the byte version Color8
is that the former is more flexible, so it would make sense to be the "default" choice. However, the other way could make sense, but not as Color16
, it would be Color32
(or maybe it would be better to call it ColorF
, or something else).
Sorry yes Color32. ColorF also works and there's some presedence for that.
it would make sense to be the "default"
I'd say the other one should be default. Working with pixels quickly adds up to a lot of memory. In most cases quadrupling your memory footprint without realizing it when you most likely won't need HDR seems misleading. I'd rather you force the user to think about that by opting in with the non-"default".
@dotMorten System.Drawing.Color has a really nasty memory inefficient implementation as noted in https://github.com/dotnet/runtime/issues/48615 . So another one is needed.
If this issue can be acted on before dotnet6 then it will save a lot of duplication because Maui.Graphics and Maui can then depend on it (link).
I agree with @dotMorten that 32bpc is excessive for a non-specialist Color-type. The consensus of this thread is that specialist color types belong in other libraries. Maui.Graphics currently uses a 32bpc Color type but that should be at most 16bpc.
The consensus of this thread is that specialist color types belong in other libraries.
I disagree with that conclusion. A "specialist" 32bpc Color type has a very wide range of use cases, so I disagree that it's actually specialist, and I think it's worth including inside of .NET alongside an 8bpc type. Otherwise I agree, System.Drawing.Color should be replaced, and if we want to call the 8bpc one "Color" then that's fine with me as long as we have a 32bpc one available.
A "specialist" 32bpc Color type has a very wide range of use cases
Can you give them? Only HDR has come up. And HDR is not only a niche case, but a niche within photography, and it doesn't make sense to support photography usage without a lot of extras - color profiles at least.
Just my 2 cents:
Cons:
Pros:
Compromise I come from a county with 17 elected political parties, so compromise is in our blood. Maybe this can be implemented as an one or more extension packages? Just like sql-server support for EF-core is in a separate Nuget package.
.. why the System.Numerics namespace? Not that I have a better suggestion
How about System.Graphics.
"DirectX and System.Drawing use ARGB
DirectX does not explicitly use ARGB, it uses all types. I use R8G8B8A8_UNorm (RGBA) typically for swapchain.
The major graphics players in the .NET ecosystem all have their own types already and this will not provide unification among them
If they choose to not adopt a communicable type, that's up to them, meanwhile, the up and coming graphics libraries - which we are in dire need of, could use this type. And anyway, system.numerics was generally adopted.
Just for clarification, however, would you feel the same way about having some "color" functions that operate on Vector4 exposed under the System.Numerics namespace (rather than exposing new types)?
Isn't this just an admission that we need a global color type, but instead of having a color type, telling people to use Vector4?
Vector4 also comes with the benefit of being recognized as a JIT intrinsic so it can be passed in a SIMD register automagically and has lots of useful math already defined.
I think one of the points of Microsoft implementing a color type would be to provide SIMD where appropriate to a natural representation. Which would be preferred over the current situation where people are implementing their own color types (myself included) that certainly don't have any optimizations.
Wider color gamuts and higher dynamic range will become the norm over the next few years, so by the time those additions would be useable by any number of developers, they'd be on their way to obsolescence.
I've just upgraded to a 10 bit / HDR600 display myself (up from 8 bit / HDR400) And yeah, from now on, my own efforts will equally consider that richer scenario. But don't RGBA(4byte) and RGBA(4float) handle all cases anyway for regular color handling - just putting aside the legacy formats BGRA / ARGB for the moment and the many variants of color information that can be useful in DirectX - the 4 float RGBA can handle any HDR or X-bit color precision (10 -12 bit) and the 4 byte RGBA is still useful for 8bit textures and vertex colors which don't need to be replaced, as 8bit textures can still be processed by 10bit / HDR pipelines, can't they?
Any color/pixel struct would implement interfaces that provide information about the pixel format, memory layout, color space, etc.
How does that work? I mean, I'm just not experienced enough to understand how you will use interfaces to describe formatting and color info. I would imagine if the struct is primitive and can be interpreted in different ways, then there might need to be a color-descriptor associated with any representation. Example,
class Texture<Color32>
ColorDescriptor descriptor; describes color space etc
Color32[ , ] data;
You just can't include all the information in the color struct, and you don't want to.
The idea of calling the float version Color and the byte version Color8 is that the former is more flexible, so it would make sense to be the "default" choice.
Flexible maybe, but I wouldn't call it common or default. The common case is 8 bit textures, that's where people will start - loading and working with bitmaps and even when you get to feeding shaders, you're still using 8bit textures. Vertices will typically be defined with 8 bit color as well. Now that I actually have a 10 bit display I might start trying out something higher precision, or maybe I won't, maybe I'll just keep using 8 bit textures and leave it to the shaders to bring out the higher range of colors.
As discussed in https://github.com/dotnet/runtime/issues/14825, the existing
System.Drawing.Color
type is not ideal for all use cases and .NET Core could benefit from new color types.This proposal includes two new types:
Color
andColor8
, in theSystem.Numerics
namespace in a new folder (library? package?) calledSystem.Numerics.Colors
.Rationale and usage
The new
Color8
type uses fourbyte
values. This is intended to be the computationally-cheap color type. It is fairly similar toSystem.Drawing.Color
. TheColor8
type is not suitable to use for HDR color math, including on high-end displays or in VR, but theColor8
type is ideal to use on computers which display 8 bits per channel, without having any unnecessary bits. A color type using bytes can also be accelerated on GPUs when used on vertices.The new
Color
type uses fourfloat
values. This is intended to be the fully capable color type, even if it's slower thanColor8
and takes up four times the space in memory. The reason this is calledColor
with no suffix is because it should be presented to users as the default choice, since it is the most capable. This type can be used for HDR color math, since it supports values outside of the normal [0, 1] range, and it supports much higher color precision thanColor8
. HDR implementations typically need 10, 12, or 16 bits per channel, floats have 23 bits of mantissa which is plenty. A color type using floats is also ideal when exposed to shaders (GLSL or similar).Both of these types have properties for setting with the other's member type (
R8
/G8
/B8
/A8
/Rf
/Gf
/Bf
/Af
), as well as operators and constructors to convert between them, so it is easy to interchange these types, including the ability to do high precision color math and store it in a low precision (byte
) context, or vice versa.Additionally, this proposal includes
H
/S
/V
properties for getting and setting hue, saturation, and value. This is a very common workflow for colors, especially when getting values from user input.Finally, there are
Colors
andColors8
static classes which have preset colors. The existingSystem.Drawing.Color
places these as static members of that struct, but it is quite useful to have these separate to distinguish them from other static members (such asFromHSV
and any others we wish to have).The uses of these types are too broad to include specific code examples. See the above text for information on how these types are expected to be used.
Proposed API
None of the properties listed below are auto-properties, all non-static properties read/write from the fields.
The proposed code for this API is collapsed, just expand it to view the code.
Details
While making this proposal, I referenced several open source color types:
Of course,
System.Drawing.Color
(see docs). Some things fromSystem.Drawing.Color
were not included or implemented differently. For example, instead ofGetHue
etc methods,H
etc properties were used.Godot's
Color
type (see non-C# docs), MIT license. It usesfloat
RGBA, and is very similar to the proposedColor
type. Godot's strategy is to have one universal Color type that's used everywhere.Stride's color types (formerly Xenko), MIT license. Stride's strategy is to have tons of types and write tons of code for every use case, and there's a point of bloat and diminishing returns here. Stride has
Color
as using fourbyte
values andColor4
as using fourfloat
values, among several other types, but this isn't ideal and they have expressed that they're open to changing this.The proposed
Color
type is also very similar to MAUI'sColor
type, which uses floats, has methods to convert between byte and HSV representations, and has a static classColors
with preset colors. It's also similar to Unity'sColor
type, though this one is closed-source.When it comes to internal memory representation, string parsing, and importing from
uint
/ulong
, these types use RGBA order. Traditionally, Microsoft APIs such as DirectX andSystem.Drawing
use ARGB, but the rest of the world uses RGBA. Using RGBA order everywhere helps fit with the goal if "Semantic parity with CSS".Open Questions
Are there any common use cases that these color types are unsuitable for?
Are there any other methods or properties which should be added?
What are some good test cases to add?
Pull Request
None so far, but see https://github.com/dotnet/corefx/pull/40733 for a closed proposed (and outdated) implementation.
Updates
None so far.