Skybox in shader vs in lighting.

[holdingjason]

Right now we are setting the skybox in the shader. This works fine for static skyboxes but curious how this would work with something more dynamic like AzureSky (which we use) that is set and adjust on the fly in the skybox material under lighting. Since the skybox is updates constantly, ie sun moving across the sky, clouds etc would be interesting to have this effect the ocean as well. I will try and do some research into this as I have seen this same type of question pop up in other ocean packages.

[Midda-C]

Hey! We've got a similar situation on our project, too. We've partially addressed this, but it's still not totally workable for our setup yet, and we need to look at it more some time soon.

We're using TENKOKU Dynamic Sky from the asset store (with some modifications). In the #4 dynamic lighting issue (specifically this comment here), I mentioned how we're currently dealing with this. We just have a reflection probe as a child of the ocean prefab (but it can be anywhere in the scene, really), set to render nothing but the skybox. It has a basic script on it to save its output to a render texture and apply that to the specified ocean material.

A few things. We currently have the reflection probe set to update every frame, which isn't super expensive given that it's not rendering anything but the skydome, but it's still overkill. The time of day doesn't change that rapidly, so a more efficient approach would be to set the probe to update via scripting, then peridocially update it as needed.

But this still isn't an ideal solution for us. For one, the way Tenkoku works, the clouds aren't actually part of the skybox, but an object in the scene (plus you need scripts on the camera you want to render them in), meaning the probe doesn't capture them. Also, while it works well enough during the day, it's pretty broken at night. The moon and stars are again objects in the scene requiring camera scripts, so they don't get captured by the probe. Also, a lot of the actualy night sky colour comes from the light scattering in the fog, without which it's is basically just black. The end result is that our ocean at night just has a black texture as its fresnel.

At the moment, I think our simplest way to deal would be to just have a texture we set the colour of with a gradient we drive over the time of day (we already do this for various other things like the light and fog colours), so we could just manually set the night colour to what looks good. That along with Unity's screen-space reflections might be enough to get a nice result. However, one thing our current solution does better (at least during the day) is that the skydome has a nice colour gradient from the top to the horizon, which really helps the ocean blend nicely into the distance.

Of course, planar reflections as discussed in issue #31 would give the best results, but that's far too expensive for us.

As you said, this is a problem that pops up in most ocean packages. There may not be a one-size-fits-all solution, as it's going to come down to the specific needs of a given project.

[holdingjason]

Thanks that helps @Midda-C . Will give that a go on Azure and see how it looks.

BTW what project are you working on, if your letting the public view it yet that is. Just curious always interested to see how other as using the same tools.

Thanks

[Midda-C]

I'm glad that my rambling is helpful in some way!

I'd love to show a bit of our project, but we're currently in talks with a publishing partner, so we're unfortunately not really able to share much publicly just yet.

I'll be sure to share when I can, though! :D

[holdingjason]

No problem. Please do and good luck.

[huwb]

That's disappointing that the skybox is causing you so much trouble @Midda-C . Now that we've won a bunch of battles with other crest issues/features over the last weeks i think i'll have some headspace to ponder this and I'll post back with thoughts later. perhaps we can discuss here.

[Midda-C]

No worries. I believe it's a pretty common sticking point with most ocean solutions, given the age-old problem of anything with reflections in realtime.

Perhaps an easy workaround could be the option to choose between either the current skybox texture, or a couple colours (one for the horizon, and one for the sky, making a gradient) which can be driven by the user?

[holdingjason]

That gradient idea is not bad. Would not pick up any of the clouds etc but would probably be a decent work around. Right now messing with outputting a cube map from the sky system ie AzureSky and then assigning that on the fly to Crest via the shader ala your method @Midda-C.

That might work but the ocean does seem very sensitive to darker skyboxes. So not sure that works great.

[Midda-C]

Cool, I hope you get some usable results from that.

Yeah, a gradient isn't a very robust solution, it came to mind because I remember playing with Suimono water, and it had the option to just pick a reflection colour, which works okay if you don't have a lot of detail in the sky. If you wanted to see the clouds, you'd still need something like screen-space reflections on top of that.

[holdingjason]

Well at least with a gradient you can get the nice sun to dark effect that will at least approximate what is going on in the scene. Also lets you control the look without being totally dependent on the sky as I said it sometimes makes the water look to washed out etc.

[huwb]

Thanks guys. I'm not sure whether to go to code with the gradient idea - the alternative would be to just author a low res env map texture with the desired gradient in it which would work with the existing code and not require special code. It might be trickier to animate though if its baked into a texture.

Is that the idea - so you get three or four colours and you'll drive them with an animation that is keyed on the time of day? With the idea being that this is much more efficient than rendering a dynamic reflection probe?

[holdingjason]

Yes that is the idea at least for me anyway. May allow for more direct control vs being dependent on the actual scene/skybox etc. Sometimes that is to bright or to dark so you don't always want it to completely match what is going on in the sky but more of a approximation or suggestion of what is going on in the sky. When I have tried doing either the reflection probes or creating a cubemap from the sky on the fly the water always looks washed out and loses a lot of its definition, right now I still use the standard skybox you provide because that is the only one that looks decent. Need to do more research into that in case its more of an issue of how we are applying that cubemap/lighting to the ocean. So honestly not sure gradient is the way to go because maybe there is other things at play that are the problem.

[Midda-C]

Sorry, completely missed the new responses in this thread.

Is that the idea - so you get three or four colours and you'll drive them with an animation that is keyed on the time of day? With the idea being that this is much more efficient than rendering a dynamic reflection probe?

More or less, yeah. We'd be fine with our current reflection probe hack if it didn't completely fail at night time (not because of Crest, just because the probe misses all of our screen effects and just returns black). The gradient concept was just the first thing that popped into my head, so I'm not pushing it as a well-researched solution. :P

But, i think it'd do the job? Unity does have some built-in gradient stuff, but I've never seen it used directly on a material before, so I don't know if it can.

[huwb]

I think it would be pretty easy to try a simple gradient, even if it's a little hacky at first.

I wonder though if it will need some kind of heading/yaw/azimuth component as well, not just the altitude component. Without that you would get the same colour no matter what direction you face and where the sun/moon are.

I guess the logical extreme here is we are generating procedural sky colour, similar to the procedural sky in out of the box unity.

That feels like a rabbit hole though - if we have a specific idea of something to try then lets give it a shot. I start to doubt there is a one size fits all solution that works for everyone but we might get something that works for a good set of cases.

[huwb]

Hi guys, I've thrown in a simple gradient test to get the ball rolling - see the above commit and the new toggle and options on the ocean material shown in the screenshot below. The branch is feature/sky_gradient .

Oh for the handle values, the units are not degrees or radians - for now (and for maximum code efficiency) they're just the fastest thing i can do in the shader - look at the y component of the reflection vector. 0.0 is horizon, 1.0 is straight up (90 degrees), and 0.5 probably goes up to around 60 degrees - so its not evenly distributed atm.

[Midda-C]

I just had a look, and it's definitely giving better results at night, it's just a matter of feeding in good colours.

You're right about the azimuth idea, though. I managed to pretty easily match the colours, but we get a nicer result during the day because the skybox isn't a constant value in every direction. Here's an example (Open them each in new tabs to easily compare):

Gradient Sky Probe Sky

Not that the gradient doesn't look good, but the reflection probe does achieve a more natural result. But again, it does work better for us at night, so it's an improvement for us—especially if we can manage some way to seamlessly transition from the texture map to the gradient.

As an aside, I also noticed while testing this that the foam bubbles seem to be emissive? Or they're not taking the light into account. It's all glowing for us during the night.

[huwb]

Ahh thats actually pretty cool!! I think i "get it" now.

I know what this needs - ANOTHER GRADIENT :) . This is hacked in but i duped the gradient and added another one which is driven by the azimuth (so the handles are 0-1 values which are rotation around the y axis). The two gradients are added together to give the final result.

Its all super hacked in and i dont know if this will be manageable for you etc but thought i'd throw it over to you.

[Midda-C]

It's gradients all the way down!!!

I was able to get a pretty nice result with this. It wasn't identical to the probe, but it looked just as nice—with the advantage of being more customisable. However, it was pretty fiddly to configure. I expect trying to manage it over a dynamic time of day, and under different weather conditions/biomes might be a bit of a nightmare.

Now, I'll preface this with a statement that I have no idea how simple or difficult this is from a mathematics viewpoint—but could the vertical/horizontal position of the azimuth be determined programmatically from the light direction? The way I'm thinking about using the gradient would be something like:

• Vertical gradient essentially as it currently is. Pick a few colours (maybe choose how many colours you want to use, between 2 and 4?), and adjust their respective altitudes.

• Replace the Horizontal gradient with an azimuth colour, and a value to determine its falloff.

• Magic maths to blend it all together.

How does that sound? Does that seem feasible, or is that just madness? It feel like that'd be much easier to drive, as you could potentially have as few as three colours to pick.

Again, I'm just stating this from a user perspective, I don't know if I'm requesting something that's much more complicated than it seems. :P

[huwb]

Oh haha yeah I completely blanked taking the sun direction into account!! Actually this inspired me to try a different model that has less colours and should be much simpler to tweak - it sets the colour based on angle compared to the sun.

Also i made the colours HDR for it so you can give them any oomph they might need.

I just pushed a commit that has the "gradient sky" disabled and a new "dot prod sky" enabled, it should be fairly simple to tweak:

[Midda-C]

Yeah, that's muuuch easier to manage! It's pretty easy to—if I want to—very closely match the output from our reflection probe, plus then it's easy to take some creative license with it.

Of course, I can't speak for all use-cases, but I think this will suit our needs nicely, and will plug into our dynamic day/night system really easily. It's certainly more robust than things I've seen in other Ocean packages.

Nicely done! :D

[huwb]

Alright! Seeing that seems to have gone down really well, i cleaned it up and merged it into master. It is a material option that is disabled by default. Let me know how you get on it with it.