Dynamic Lighting

[Midda-C]

Hi again. I've just been looking at your latest changes with transparency, subsurface scattering and wave attenuation. It looks awesome!

I'm curious about the way you're handling lighting in general. From what I can see, the ocean surface doesn't appear to react to the light colour, intensity or direction. Is this an intentional choice, or just something you haven't gotten to changing yet?

Also, regarding your recent improvements, would it be possible to have some way to adjust these settings in the material? E.g. the transparency depth, subsurface colour, shallow colour, etc.

In our project, the time of day is dynamic (my reason for asking about the lighting), and we also adjust various environmental properties for different biomes (why we might like to change the surface/subsurface colours, etc).

Again, awesome work!

[huwb]

Hey! Yeah I'm completely off piste with the shading, it's just some simple terms hacked together. That's me being lazy and writing research code instead of a production quality shader!

I'll look now at moving it to the standardised PBR surface shader stuff. Assuming you're using Standard materials and PBR workflows you should get the same response out of the ocean material as everything else in the scene (if all goes well). I'll see if i can key the subsurface stuff in to the lighting as well if poss. I'll also expose tweakable stuff in the material.

I think i have all the pieces i need, so i'll muddle through it over the next few days and let you know if i make progress.

[Midda-C]

Good stuff. Yeah, we're using a (slightly stylised) PBR workflow, so a similar response from the water surface would work well.

Looking forward to it!

[huwb]

I've been messing around a bit, learning about how to make shaders properly :). I found out how to stop the material getting instanced separately for each ocean tile, so now you can edit the global Ocean material at runtime, which is very useful! I also exposed the main parameters that affect the appearance of the current shader and pushed this to master.

In the meantime i started converting the ocean material into a surface shader, WIP stuff is in the surface_shader branch. I think I've run into a blocker - as far as i can see its not possible to get a watery appearance from the out-of-the-box surface shader. I think the main issue is fresnel is not exposed. I think the way forward is to dig up the surface shader code in unity and copy paste it and modify as necessary, which is pretty nasty. I'm starting to wonder if it will be easier to make the shader I have plug into the scene lighting. Anyway i'll dig more later and post back if I make progress.

[Midda-C]

Hi mate, just curious if you've had any luck lately?

[huwb]

Hey!

I've not unfortunately. We're just about to release our latest project at work, so i've been throttling the amount of personal project stuff I'm taking on in the evenings. However I should be able to find time here and there.

Something that would help massively is to make it a bit more concrete and have an example to work with, I'm not an expert in lighting in Unity. If i work with the standard lighting setup (one direction light and the standard unity sky), will that be representative of your use case, or is another lighting setup i should look at?

[Midda-C]

No worries at all, hope your work project is going well!

Yeah, our lighting setup is pretty standard. We're basically just using a single directional light for our sun, and another one at night for the moon. I expect this would be the most common way to light a large outdoor area with an ocean. We're not using the default Unity skybox, we're using one form the Asset Store called Tenkoku.

What information would you be using from the skybox? Is that used for the reflected colour/fresnel?

[huwb]

Yes, a primary use would be the reflected colour, driven by fresnel. I saw the Unity Standard Water stuff sets up a camera and renders the skybox. This should Just Work(tm) and will elegantly captures any reflecting objects as well. While I don't love adding more render passes, it should be easy to set up and I'm considering adding it as a supported option. This would then support any kind of skybox in theory which is convenient and easy. The stuff in there currently just samples a static cubemap to get the reflection - this is the cheapest and simplest option for reflections, but perhaps a bit too constraining for dynamic time of day?

The other part is the light that is scattered around the water volume and then emitted out of the water towards the viewer (the blue/turqoise). A question is whether this should be driven by the lighting environment, or just hand keyed. Since doing volumetric subsurface scattering is too costly, driving it from the lighting would be some kind of heuristic/approximation. It might work well (I will have a play). If I don't get good results then this colour could be tweaked by hand and then blended manually using the day night cycle. This is the approach that I've used on previous projects, and its useful for non-photorealistc games where you might want to key more light from the ocean surface than would normally be emitted (I was watching some lovely footage from sea of thieves with the ocean glowing at night time which I think is an example of this).

I think ultimately there is quite a lot of flexibility/options here, I'll see if i can make some examples in the next week or two.

[Midda-C]

Obviously I can't speak for all use cases, but I think for us, being able to simply set the reflected colour (which we could then change over the day/night cycle) would probably suffice for our needs. As you said, rendering reflections from a camera looks nice and is versatile, but I think that's quite a bit of overhead for a lot of projects (or for us, anyway).

I was actually going to mention the subsurface from Sea of Thieves, because yeah, as you said, while it's emitting more light than it technically should, it really looks fantastic, especially with a stylised aesthetic. That'd definitely be a nice feature to have, and again, is something we could easily adjust for different times of day and different biomes.

[Midda-C]

Oh, something else which just came to mind. In the Unity Post Processing stack, there's a screen-space reflection effect which works pretty well. I'm unsure what property defines how reflective a surface is with this effect (obviously Unity's standard shader works with it), but leveraging that could help with the surface shading too.

[huwb]

Hi there, I implemented something really dumb which may actually work - i just multiply the ocean colours by the main light colour. The light colour is currently white so it makes no visual difference right now but if you change the colour the ocean seems to respond fairly well. It's hard to say for sure with that bright blue env map though. Is it something you could evaluate if/when you have time?

I haven't implemented being able to set the reflected colour. This would be somewhat equivalent to setting a custom env map - in the extreme case you could assign a small (e.g. 2x2) constant colour texture, but i think gradients etc help the look a lot. I don't love how the current env map looks, even with the clouds the reflection looks too uniform (to me).

Good call out regarding the post-proc stack. At some point i aim to experiment with it, and find out what is required to activate SSR. I've not made any progress on this front yet. I hope to look at it within the next 2 weeks.

[Midda-C]

Haha, that's sort of funny, because I was wondering if the colour could simply be multiplied by the light colour/intensity. I didn't say anything though because I thought that might sound dumb (I'm fairly new to shader writing). Guess not, though! I'll be sure to check it out this coming week.

Only thing that I expect would be missing with this approach is the specular highlights from the light source (particularly with things like sunsets). That's another thing quite nice with the ocean surface in Sea of Thieves, there are nice little glinty hotspots from the sun/moon.

Thanks again, will let you know how we go!

[huwb]

Hey there!

On the specular highlights, ideally these would come entirely from a HDR sky (the sun should be represented in the image with large brightness values, and the reflection of this should give the highlights). However I was unable to get this to work well after messing around with a few HDR textures i could find online. Perhaps the intensity of the sun is capped in these so you miss the nice response. With a proper HDR setup, i think the shader should work as is.

In lieu of a proper HDR setup, i eventually gave up and added the (static) option "Add Directional Light" to the ocean material which will manually do specular lighting, with options to boost intensity and change falloff. I've turned it on by default. I'm no artist but i think its roughly the right approach and is working pretty well.

I believe this latest commit resolves this issue - the ocean is now lit from the lighting setup (all driven by the main directional light), with some added flexibility around specular highlights. I did not get further with SSR, I think I will leave this as a next task.

But let me know if you have thoughts, or if I'm missing something that is still not addressed.

[Midda-C]

Hi again!

So I've just been playing around with the latest changes. The specular from the directional light works well so far. I haven't messed with the settings much yet, but it seems to behave as expected. :)

I get what you mean about the specular just working as-is with a HDR cubemap. Unfortunately, because our project has dymanic time-of-day (as well as weather conditions), this means we don't use a texture for the skydome. I wonder though if there were some way to use the output from a reflection probe? That'd have the necessary information in our case, but I have no idea if it's straight forward to get the output from a real-time reflection probe (it's easy if they're baked).

I did notice though that the foam always seems to be completely lit, regardless of the lighting conditions.

[Midda-C]

Okay, so regarding the reflection probes, we've just put together a script which goes on the reflection probe and allows us to plug its output into the Skybox of the material. That seems to work pretty well for us. I don't know if it's too specific of a use-case to bother implementing into the shader on your side, though?

One other thing I've noticed regarding lighting. You've got a few toggles there now which are handy. Would it be easy enough to have a toggle to multiply the subsurface by the light colour? It might help the subsurface look a bit more natural, but I don't know if that's as simple as just multiplying the colours.

[huwb]

Thanks for the notes about reflection probes. I've not really used these in unity so i'll have a play with them later.

That sounds cool about feeding the reflection probe into the ocean, if you're up for sharing the code I'd be curious to see if I can incorporate it in a modular way. I'll be interested to know if it can support blending between reflection probes.

Foam colour - hm yeah.. i tried multiplying it by light color but it comes out a bit too saturated. Could you perhaps try replacing this: col = lerp(col.xyz, _FoamWhiteColor.rgb, whiteFoam); with this: col = lerp(col.xyz, _FoamWhiteColor.rgb * _LightColor0.rgb, whiteFoam); And let me know how it looks for your setup?

Finally about the toggle, actually the subsurface should already multiply with light color, if you change the colour of the main directional light the ocean should look different? I just tested that it works here. There was however a bug that this multiply would not happen if subsurface is disabled - this is fixed in 28beb186e9c59535eec31206e47ce70339a958a5 .

[Midda-C]

What we're doing isn't really how reflection probes are intended to be used, but in my limited experience with them, they're a bit of a black box. I've just popped a realtime reflection probe in the scene (updating every frame for testing, but you can manually update them via a script) set to only render the skydome. We've got a basic script on the probe which just directly feeds the output from there into the 'Skybox' parameter in your Ocean material

void Update ()
{
    if (m_Material != null)
    {
        ReflectionProbe rp = GetComponent<ReflectionProbe>();
        m_Material.SetTexture("_Skybox", rp.texture);
    }

}

It's just a hacky test to see if it works, which it seems to.

I made that change for the foam colour. It's not perfect, but it's (at least in our case) a lot better than just leaving it fully lit. I noticed that during our day/night transition, there's a short period where there's very little directional light and we rely on ambient a lot. At this point, the foam is just black, so it might need to have the ambient light values added to it?

And yes, sorry, you're correct, the subsurface is taking on the light colour. I must be blind. :P

[Midda-C]

I don't know if this is a bit out-of-scope, but I was thinking about why the foam looks too saturated at sunset/sunrise. After just looking at some sunset beach images, I think it might be related to the entire foam surface being uniformly affected. Because real foam has some height, you get a bit of shading on it.

Sunset foam example 1 Sunset foam example 2

You get bright saturated colours on the sides facing the light, but shadows/ambient on the opposite side. Maybe a normal map (and some PBR roughness) for the foam might help give some more believable shading? Again, I don't know if that's a bit out-of-scope, or if there's a simpler/better solution. Just writing down my brainfarts. :)

[huwb]

Yeah good call about the ambient. Could you please try this: col = lerp(col.xyz, _FoamWhiteColor.rgb * unity_AmbientSky.rgb, whiteFoam);

In my project, the ambient is set very dark and it looks awful, but im pretty sure thats me failing at lighting, perhaps?

As a next step, we could mix in a bit of the sun light color as well, perhaps? col = lerp(col.xyz, _FoamWhiteColor.rgb (unity_AmbientSky.rgb + 0.2 _LightColor0.rgb), whiteFoam);

Let me know. It's non-ideal that i dont have a proper lighting setup - its blocking me from experimenting with this myself. i need to take some time to do some outdoor lighting tutorials.

That's great ref! I'd love to give the foam some depth and do NdotL lighting, as you hint towards, and i think its doable. I came close to doing this for previous projects but never found the time to explore it. It's high on my list, i hope to get onto it soon.

(off topic - you and dizzy have now made significant contributions to Crest and it's super-appreciated. if you would like to be credited on the landing page, let me know. if so, let me know how you'd like to be credited - name, or user id, etc).

[Midda-C]

In my understanding, the scene's ambient light is added to the surface along with the regular surface lighting, not multiplied. Ambient values are usually fairly low, so multiplying means you just end up with the final lighting output multiplied by a value of like 0.3 or something, making it very dark.

I tried modifying that line by adding unity_AmbientSky.rgb rather than multiplying, and it did fix the foam going so dark at the day/night transition periods. It still suffers from the saturation issues you mentioned though, I guess just because it's a uniform surface all going the same colour.

Hah, I don't feel like I've really done anything to warrant a credit. :P Dizzy's actually been poking around to get Crest going in our project, though. Hopefully soon we can actually show you how it's looking in our title!

[huwb]

I thought about the ambient. I think it should actually multiply. The "Vantablack" material will appear very black, regardless of ambient light, because its albedo is very low. Foam should have a high albedo value i think, but it should still be a multiply.

I really need to get a proper lighting setup. Then i should be able to actually propose solutions for you instead of asking you test random hacks :). That's next on my list.

I'd love to see it running in your title, please do share if you get that far!

[Midda-C]

D'oh, sorry, I'm not a coder, so I worded that incorrectly. Regarding ambient, I think it should be done as follows (please excuse my programming-illiterate pseudo code):

albedoColour * (lightColour + ambientColour)

I messed up before by saying ambient should be added to the surface, I meant it should be added to the light. Multiplying the ambient and the light results in the light information being very low, since ambient is generally very dim in comparison. Basically, the surface should still be visible if you have no lights in the scene.

For context, our sky midday ambient RGB value (0-1 range) is 0.04, 0.33, 0.39, so if that were multiplied with the light, it'd never get very bright.

Again, my shader experience is pretty limited (and I use node-editor training-wheels instead of actual code), so there's likely a lot more to it than that, but I think that's the gist of how ambient is applied in Unity.

[Midda-C]

I guess it'd be more useful if I showed you examples. Here are how the two changes you mentioned look in our scene (midday lighting).

_FoamWhiteColor.rgb * unity_AmbientSky.rgb

_FoamWhiteColor.rgb (unity_AmbientSky.rgb + 0.2 _LightColor0.rgb)

So the second one is better, but considering the foam has a completely white albedo, under full sun, it looks a bit dirty. I assume that's due to the light being at only 20% strength.

[huwb]

Sorry it took me so long to get my head around this. I stupidly had not generated lighting before for the example scene which was making everything funky. I should be able to make things converge a bit faster from now on.

I'm now using ambient light colour to light the water and the foam. The subsurface light is based on the primary light. I'm using the constant SH coefficient to give the ambient now - unity_AmbientSky doesnt seem to give me meaningful data for some reason.

Not super stoked on using the ambient light colour to light the foam. I could mix in some of the primary light, or try the NdL lighting as we discussed before.

Anyway besides foam it looks pretty good for me now - for my example scene at least. I'm not sure if it will work for you though. Next time you guys get latest could you let me know? Thanks!

[Midda-C]

Yeah, just got latest, I can barely see the foam in my scene now. It doesn't really look like it's taking the primary light at all. I modified the foam colour to do the following:

col = lerp(col.xyz, _FoamWhiteColor.rgb * (_LightColor0 + half3(unity_SHAr.w, unity_SHAg.w, unity_SHAb.w)), whiteFoam);

Which seems to work better (and made the same change to the underwater foam). I wonder if part of the issue is how the ocean itself looks so believable, like an actual physical substance, while the surface foam is just a flat uniform colour over the top. I think the subsurface foam works really well, it's just that the surface lacks the same feeling of texture that the rest of the ocean has.

I don't think it needs very much, though. To be honest, when using the primary light colour, it's already basically the same as what Sea of Thieves does with its foam (just need the ability to tweak the various thresholds and such as to when and where to generate foam). Maybe some NdL lighting would be enough to really give it some oomph.

[huwb]

Yeah looks good to me - i went with that! I added in direct light for both white foam and bubbles and retweaked the ocean material slightly in my scene.

I'll make a new feature request to revise lighting model for foam.