Xovval
commented
2 years ago Maybe decoupling in HDRP is useful?
Decoupling the visual environment from the lighting environment
You can use the sky Lighting Override Mask in your Unity Project’s HDRP Asset to separate the Visual Environment from the environment lighting. If you set the Lighting Override Mask to Nothing, or to a group of Layers that have no Volumes on them, then no Layer acts as an override. This means that environment lighting comes from all Volumes that affect a Camera. If you set the Lighting Override Mask to include Layers that have Volumes on them, HDRP only uses Volumes on these Layers to calculate environment lighting.
An example of where you would want to decouple the sky lighting from the visual sky, and use a different Volume Profile for each, is when you have an HDRI Sky that includes sunlight. To make the sun visible at runtime in your application, your sky background must show an HDRI sky that features the sun. To achieve real-time lighting from the sun, you must use a Directional Light in your Scene and, for the baking process, use an HDRI sky that is identical to the first one but does not include the sun. If you were to use an HDRI sky that includes the sun to bake the lighting, the sun would contribute to the lighting twice (once from the Directional Light, and once from the baking process) and make the lighting look unrealistic.
Right now, when entering the Varjo AR video-see-through, the directional light is not adjusted to the room automatically. Thus, the ambient light (cubemap HDRI) has a different light source than the direct light (shadows & highlights).