Geenz commented 3 years ago

(Disclaimer: this issue is being created for general public tracking of a feature under consideration for development. Please do not make any assumptions about the viability or likelihood of this feature being released any time soon. The same applies to any features that would be dependent upon this landing first that may be mentioned below. This is here to discuss rationale, risk factors, potential content impact, possible outcomes, and to give a more focused thread of conversation to the feature being outlined below.)

Is your feature request related to a problem? Please describe.

As it stands today, lighting is "kind of sort of linear" in Neos. This is due to Unity's built in PBS system which requires some semblance of gamma correction. However, there's still some things that aren't quite calculated in the proper color space- such as light falloff and ambient occlusion. Although this is isn't a problem in and of itself depending on one's stylistic choices, this will only become further complicated down the road as additional graphics features are added to further enable said stylistic choices by users.

Relevant issues

63 #1561 #1423 #1192 #619 #1760

Describe the solution you'd like

It would be good to have a general color management solution inside of Neos to better enable a wide variety of use cases in terms of enabling users to "pick a style" and author to that style.

Describe alternatives you've considered

Adding more graphics features that are gonna become more difficult to maintain later without this first.

Additional context

Given the eventual completion of desktop mode, this will also enable Neos to better support HDR display output as well down the road, something that's become pretty important in the entertainment space generally.

Additionally custom shader support, if we ever support it in whatever form we may support it in, will become far more difficult to support without this landing first

Content Impact

Going from gamma to linear lighting is not as trivial as "flipping a switch" effectively. This requires deep changes to how Neos stores and handles textures - even requiring us to store color space information for textures and colors as a whole.

This has several side effects, both transparent to users and immediately obvious to users:

All texture variants will at the very least need new metadata generated for them to associate this color profile information with them.
Some may outright need to be regenerated (such as in the case of Crunched textures - where color profile impacts how the texture gets compressed)
Things like light falloff will look brighter in many scenarios, and darker in others due to all parts of the lighting pipeline becoming "linearly correct"
Light intensity also plays into the above content impact generally if we assume all light intensity values are in a "linear space"
Shaders like Xiexe will also likely be impacted due to some of their calculations implicitly not being in a linear space, likely requiring some quick fixes prior to any release being considered for this feature

Potential Outcomes

If gamma correction is introduced into Neos' pipeline, you can likely expect at least one of the following outcomes depending upon community feedback and internal testing.

Full linear lighting - everything is linear without any option to disable or otherwise modify its behavior. This is the fastest path to the feature, and has the largest immediate content impact.
Partially linear lighting - most things (particularly anything that touches PBS) are linear, with some tweaks to maintain as close to existing behavior as possible for other things (like Xiexe). This is perhaps the most likely "near term" outcome.
Full linear lighting with some controls - everything is linear with a few parameters to adjust the look and feel of objects and lighting, with existing content being assumed to be in "gamma space". This is the longest term outcome given it would require some relatively deep changes to how we handle a wide variety of things such as light sources and certain shader values (and likely require some hacking about on the Unity side - definitely not something we want to do).

Geenz commented 3 years ago

Some questions and answers given the above that have already been considered:

Q: Should all existing maps be treated as gamma space?
- A: No. Normal, lerp, occlusion, and metallic maps should be considered to be in a linear space and interpreted as such. Existing albedo, any Xiexe shading ramp maps, and so on should all be counted as being in gamma space. All maps (old and new) will have the ability to explicitly set what color space they should be sampled in (likely just being sRGB or Linear in the beginning).
Q: Can we add a "gamma slider" to lights?
- A: Tricky. This would require some hacking around in Unity's rendering pipeline to properly support - and even then it would likely only apply to deferred rendered objects. Injecting additional lighting parameters isn't always trivial in Unity without SRP, and migrating to SRP is highly unlikely given the amount of effort required.
Q: Can we add a color space or linear tick box to colors?
- A: Yes. However, HDR colors make this a bit tricky. In effect we have to apply some sort of gamma ramp if a given channel is below 1.0. Anything above 1.0 is generally not accounted for in most gamma calculations. The best we can do is correct and apply the existing scalar for HDR colors, or just assume anything greater than 1 in any channel as "linear".

kulzae commented 3 years ago

Id say its better to bite the bullet and switch now than down the road when theres more content to be affected by it

InsaneGrox commented 3 years ago