Detecting Underwater

[hayes87]

Hello, this is not actually an issue but a doubt. Im trying to figure out the best way to detect if a player is underwater and apply a shader to the part of the screen thats underneath the waves.

Whats the best approach?

Thanks

[holdingjason]

@hayes87 Hi. We are working on the same problem. We can easily detect when the camera is above or below and modify the shader appropriately but the issue is that the camera is not always fully above or below the water line so you get these spots where your half below half above and it looks wrong. Not sure yet how to solve it yet so good place to have a discussion and try out some ideas.

[huwb]

Thanks for starting this thread. I've seen some games mask it with a flash:

https://www.youtube.com/watch?v=a1ezfA2sb_Q

The look of a camera half underwater is pretty sweet but I'm not sure how easy it is to achieve as you need different post processing for half of the screen?

https://www.youtube.com/watch?v=KIcJgznQWqA

[hayes87]

Yeah, i was researching as to how the Witcher3 does this. It seems they grab a zdepth of the water, clamp to get only the line of water that crosses the screen and then some magic to create a gradient with this.

Some screenshots from renderdoc https://cdn.discordapp.com/attachments/409446549567963151/489451627246387211/unknown.png https://cdn.discordapp.com/attachments/409446549567963151/489451449592446989/unknown.png

[huwb]

Ah thats interesting. Do they produce a half-in-water effect from that? Do you have the rendered result? I looked at some youtube videos but couldn't find a good shot of the camera transitioning into water so far..

[hayes87]

Ah sure, forgot to link it https://cdn.discordapp.com/attachments/409446549567963151/489513816632066051/unknown.png

[hayes87]

ops closed by accident hah

[huwb]

From the images it seems like they darken the water near the camera and ramp down the refraction based on the gradient, and also modify the lighting under the water.

I think it would be possible in crest to generate a screen facing strip of tris along the water intersection line, and then draw "some blurry stuff" on top of it, to emulate some of the half underwater youtube video i linked above. It wont be simple, but i think it could work. It would be somewhat similar to how the crest logo is rendered on the ocean surface, but using some more shader stuff to find the intersection line and make it screen facing.

The bigger problem for me is how to achieve different postprocessing and fog etc below the water vs above. I know that this kind of thing is possible with custom tech (c.f witcher images above, and a ue4 example http://oliverm-h.blogspot.com/2014/08/ue4-localized-post-process-effects.html ) but i havent been able to piece it together in my mind for unity. However i'm far from an expert on this and I have limited experience with the post processing stack. Maybe it can be done?

[moosichu]

Uncharted 3 uses some interesting techniques to achieve this effect with its capsized boat.

https://www.slideshare.net/mobile/agebreak/water-technologyofunchartedgdc2012

The water verts are clamped to the Shape of the ship and a polygon skirt it used to achieve a fog effect. It's quite effective.

[huwb]

Yep, great point. That kind of thing seems doable.

The skirt could have a stripped down ocean shader perhaps. I guess when the camera is fully submerged the skirt becomes a full screen quad?

I'm super intrigued to see if this would look.. I'll report back if i do some experimentation on this but i probably wont get onto it in the short term as I have a few other issues i need to resolve first. If anyone else looks at this feel free to report results!

[huwb]

Ok the temptation to try it out was too strong!

https://drive.google.com/open?id=1OzgtMhRGT54fBekC0q-bFM1enT845PKu

Perf is good and besides a few edge cases with awkward view directions that need to be addressed, and a bunch of making it production ready, i believe this could be a workable solution.

I'm also able to get the underside of the ocean rendering but right now it assumes camera above water and it looks strange. It would work in theory though I believe, just needs more work.

It's pushed in a branch feature/underwater-skirt , it can be seen by grabbing this branch and opening main.unity.

It could use a brave early adopter to try it out and find issues in a real project scenario to help make it shippable. Anyone up for it?

[danilonishi]

Hi huwb, thanks for working on this. Really appreciated! Seeing you so inspired and engaged to this project makes us feel confident about the potential of your solution. We're testing it and considering using your ocean render in our upcoming game.

Keep it up! 😊

[holdingjason]

I will definitely give it a go as well. As I have shown we are doing underwater transition stuff. That transition between under and above still has these issues so need to do something (was thinking of just doing the flash but this might be way better). Only issue that I am not sure of with this would be how to deal with the changes of fog and post processing effects that the underwater needs vs above water. Cannot really apply them until your all the way under which this half/half approach would not allow so not sure on that. Maybe we will be able to use this approach before turning on the underwater effects and it will look fine just not sure. Babbling I know.

[huwb]

Thanks for the comments!

There was an issue where it only worked with near plane at Z=1 (quite a large near plane value). I've generalised it for any near plane in a fix pushed just now.

Good point about underwater postprocessing. I don't think this technique addresses the issue of potentially having different postproc above vs below the water. I think that would go back to my question above about ways to split unity postproc along a arbitrary line. Do you think not having dedicated underwater postproc would be an option - just using depth fog/caustics etc from the shader combined with above water postproc? Are there features you use in the underwater postproc besides fog?

In other news I've started experimenting with getting the underwater look working, there is an artifact due to a slight overlap i had to add to avoid cracks when you're looking up through the water (the underwater look is placeholder rihgt now, it doesnt refract properly or fog etc) - it adds an objectional strip beyond the water that I'm currently thinking about how to solve..

Thanks @danilonishi , it's been a pleasure working on it! I'd be very interested to know if you go with Crest or with something else - if you have any feedback on where Crest falls short do send it through - thanks!

[holdingjason]

@huwb Right now we do some bloom and color adjustments using post proc stack 2 but honestly we could probably live without it depending on how good we can get the underwater look using this effect. Need to play with it to see for sure. We also do some adjustments to the water shader itself to get the under surface looking nice (mostly change to a different cube). I was thinking we might also be able to fade in those effects as the camera goes completely underwater and use this to help with the transition but not sure how that would work. So really need to grab your latest off this branch and just bang on it. Just wrapping up some work on generating fish, fish flocking and movement etc.

[huwb]

Cool. It would be possible to do colour cube in this shader as well - well I've never done fancy 2018 colour cube so I'm not sure what sorts of complexities there are these days, but I think the core concept is fairly straightforward and should be something we can replicate.

I've pushed another version which has the under side of the ocean rendering, and it has a bit more test geom and has a few more minor fixes, probably best to take this one. you may notice small cracks at the fluid interface now and then, if i dont find a better solution im thinking to just render "some blurry stuff" to mask it - to emulate the fluid interface being defocus blurred by the camera. I've laid some of the foundation for this now, but haven't done a proper test yet.

[Midda-C]

Wow, this is awesome! We weren't planning to let the camera in our game go underwater, purely to avoid all of these issues. It does though occasionally clip with the water's surface, so it'll be great to not only stop worrying about that, but having it actually look really cool!

[huwb]

Nice :)

I haven't worked much this week but i did get a bit futher - video below. It draws a fake DoF mesh close to the camera to hide the cracks that can appear between the water surface mesh and the underwater skirt mesh. This can be an artist authored texture (its just a strip of alpha that tracks the water interface).

https://drive.google.com/open?id=1eIfzblVXHG1t09ml_ttB9Ip4qnWYjrmH

Still more to do - its not properly integrated into the shader code and has some issues to work through but I was capturing videos and thought id post a WIP update.

[hayes87]

Whoa this is starting to look really nice!

On Thu, Sep 20, 2018 at 6:17 PM Huw Bowles notifications@github.com wrote:

Nice :)

I haven't worked much this week but i did get a bit futher - video below. It draws a fake DoF mesh close to the camera to hide the cracks that can appear between the water surface mesh and the underwater skirt mesh. This can be an artist authored texture (its just a strip of alpha that tracks the water interface).

https://drive.google.com/open?id=1eIfzblVXHG1t09ml_ttB9Ip4qnWYjrmH

Still more to do - its not properly integrated into the shader code and has some issues to work through but I was capturing videos and thought id post a WIP update.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/huwb/crest-oceanrender/issues/70#issuecomment-423336508, or mute the thread https://github.com/notifications/unsubscribe-auth/ANUYTmpIC-uS_qYfUYrpVtdouf725zWEks5udAXzgaJpZM4WiPjw .

[holdingjason]

Great job so far. Very nice you are a water genius.

Ok so the one thing that jumps out at me immediately is the "fog". This works great for objects under the water be seems to be ignoring the water surface.

SO here is an example of what I saw.

Here the ocean stretches off forever and anything you look up at will show up as well shadowed by the ocean.

Another shot of the object disappearing but showing up still against the surface.

Here is how we would want this to look (main branch using deferred fog exp2 0.05).

[holdingjason]

As for other effects like bloom, wiggle, lighting etc I am thinking we can probably just ease those in as we go further into the water and make sure we are fully submerged or would have to add whatever we need to the shader itself or do some custom processing to the underwater skirt. I am guessing that would be possible? Thoughts on that.

[huwb]

Yeah you're right about the fog looking funky for the underside of the water surface, thanks, i'll see what i can do when i work on it next.

Yes maybe it will work to ease it in over 1 or 2m of water depth as the camera gets deeper in the water. From a quick google it looks like blending is possible so this is a good shout.

I suppose if we can make an underwater postproc fade in, then we should try to do as much as possible there so we dont have to build/replicate/maintain a bunch of code in the skirt shader that tries to replicate the postproc. Could be a bunch of work. But there is the option there of doing stuff in the skirt shader.

Good ideas, feel like we're converging on something!

[holdingjason]

Glad you saw what I was seeing with the fog. Was also seeing some artifacts with unity fog while above water as well. Will see if I can grab a shot of that and post it. I think if we can get the fog working properly below/above water then that should make this workable.

As far as post proc stuff I would agree the less we have to tweak the shader the better since that is way more flexible ie your never going to please everyone so will play with fading in those effects. Will look for blending and give that a try, if you saw something that looks promising shoot me the link.

Great job and glad we could be of help. Thanks for doing all the heavy lifting and let us know anytime you need any help.

[holdingjason]

Here is the shot with the artifacts. Just changed the fog to .025. Camera is set to deferred and using post processing 2 with deferred fog and not excluding skybox.

[holdingjason]

verified that the fog works fine in the main branch so only see the artifacts in the skirt branch.

[huwb]

i did a big refactor and rewrite of the scattering code to unify the code between underwater and above water. and i found a system variable to tell me when i'm drawing a backface (for free), so no more ugly fireflies!!

https://drive.google.com/open?id=13io-152FFI6StqCMRohb74KD8I5zEZ8x

there's still a bunch to go - it currently triggers unity to grab the framebuffer twice, when only once is needed. and there are awkward edge cases when looking straight down through the water surface and the near plane intersects the water in complex ways and the 'skirt' fails. need to think about how to address this..

[holdingjason]

Great job. That looks so much better now. Getting a few of the flickers still underwater but way less then before. Will attempt to start integrating it into our actual scene and see if I can get the look close to what we want.

[Midda-C]

The underside of the water in Subnautica is done really well. I'm a bit of a layman, but at a glance, it looks like the transparency of the surface is masked using a fresnel with a fairly sharp falloff.

https://youtu.be/kJN6A8HYTAw

[pschraut]

The bigger problem for me is how to achieve different postprocessing and fog etc below the water vs above.

I created a thread in the Unity forums, where I ask how to achieve this with Unity's Post Processing Stack v2 solution: https://forum.unity.com/threads/how-to-apply-effects-depending-on-a-mask-under-water-transition.562516/

[holdingjason]

@pschraut Thanks good idea. Ya the only thing so far I can come up with is blend into those settings as you go deeper below the water which I am guessing will work but would love to be able to apply post effects split screen. My guess you can only do it with two cameras.

[mschwarz1]

Just discovered this package and I'm really blown away by it so far. Really cool! Tried to correct the issue with the skirt failing when looking close to straight down and came up with this addition:

From UnderwaterSkirt.shader:

// isolate topmost edge
if (v.vertex.z > 0.45)
{
    // Check if looking downwards .8 just seemed to work pretty well
    if (dot(forward, float3(0., -1., 0.)) < .8 && abs(_WorldSpaceCameraPos.y - _OceanCenterPosWorld.y) < 2) {

        half2 nxz_dummy = (half2)0.;

        float2 sampleXZ = o.worldPos.xz;
        float3 disp = float3(sampleXZ.x, 0., sampleXZ.y) + up * _OceanCenterPosWorld.y;
        for (int i = 0; i < 6; i++)
        {
            // sample displacement textures, add results to current world pos / normal / foam
            disp = float3(sampleXZ.x, _OceanCenterPosWorld.y, sampleXZ.y);
            SampleDisplacements(_LD_Sampler_AnimatedWaves_0, LD_0_WorldToUV(sampleXZ), 1.0, _LD_Params_0.w, _LD_Params_0.x, disp, nxz_dummy);
            float3 nearestPointOnUp = o.worldPos + up * dot(disp - o.worldPos, up);

            float2 error = disp.xz - nearestPointOnUp.xz;
            sampleXZ -= error;
        }

        o.worldPos = disp;
    }
    else {
        o.worldPos += 1. * up;
    }
    // small fudge to lift up geom a bit and cover any cracks. it will render UNDER the ocean so any overlap will be covered
    //o.worldPos += .02 * up;
}
else
{
    // bottom row of verts - push them down a bunch
    o.worldPos -= 8. * up;
}

It doesn't actually solve the issue but it covers it up pretty well when I was messing about. I don't have much experience with shader coding so it was the best I could think up. Just thought I should post it in case anyone wanted to use a temporary solution. Cheers!

Edit: Threw on another clause because the angle it fails at decreases the deeper you go. Really displacement theoretically shouldn't be needed past a few units though hence the additional clause.

Another Edit: Refactored. Why do all the displacement work if it's not going to be used? Corrected that.

[GeorgeDziov]

Just tried underwater branch and gotta admit it looks radical! But will there be option to leave the waterline without blur?

[huwb]

Awesome to read these comments! Nice that this feature is resonating with people :).

It's somewhat parked while i tend to other things, but will most definitely be driven forwards in the near future. I'll be sure to try your changes @mschwarz1 , thanks for posting!

@GeorgeDziov yes, the hibiscus/waterline should be an alpha blended texture instead of the current blurry mess (or it could be some kind of fancy raytraced thing). It would involve sampling the texture in WaterInterface.shader frag() using the uv, instead of the procedural smoothstep thing thats currently there.

[holdingjason]

@huwb Ok finally back looking at this and the first thing that jumps out is the ocean surface now blending out with distance. I believe we had an issue with this that we solved awhile back in the main line where the ocean was not picking up fog or something.

Here is a shot of what i mean using the skirt branch. The ocean shows regardless of distance so could use fog as below or the skirt would need to blend out the ocean or something not sure on that.

This is the current main line using fog. See how the ocean respects the fog.

[holdingjason]

@huwb If this is an issue with the skirt being out of wack with main would it be possible to get an update form main to get it back in line?

[holdingjason]

Was able to cobble something together with the latest main and the skirt. This is still using fog to hide the water line and the skybox. So if I turn off fog and make sure to not exclude the skybox and you can see the results below. This is actually not optimal because the issue is again that when we come out of the water the fog is so close that the sky box is obscured and we see nothing but blue. I guess playing with this ideally we would want the skirt to cover up, fade out, the water line at a certain distance like other objects as well as the skybox.

No fog.

Fog but do not exclude skybox so you can see the skirt is not preventing the skybox from showing through in my hack.

[holdingjason]

Grrr ok I just setup the original skirt stuff again and not seeing these same issues. Need to reconstruct my scene and see how things went wrong. I will let you know once I have done that.

[holdingjason]

Ok so yes most of that was correct. You see the issue when your drop the skirt transparency down to like .1 across the board so you can see stuff below the water. The skybox is gone in this which is good (need to see if that has something to do with postprocessing which is off right now) but the water line does not fade in the distance my guess because of the transparency. So we do want to be able to lower transparency in order to see stuff but we dont really want it to effect the ocean top way off in the distance. Basically wanted the effect to work like exp2 fog does at a 0.05 setting or so. Clear up front but quickly drop off and have it effect the water as well as everything else. Have no idea how to do that honestly. The reason we are using the skirt is to get rid of the issues with fog ie coming out of the water and everything being obscured (including the skybox which is super bad but has to be done or it shows up under the water). So using the skirt makes sense to me now. Just not sure how to get it to work correctly. Must be a way to apply a exp fog like effect to the skirt instead of straight trans. Will do some research as well and let me know your thoughts from my gibberish ramblings.

[huwb]

Oh i think i know - you need to replicate exactly the settings across the ocean skirt material and the ocean material - i guess you have different density settings on these two materials? could probably write a script the replicates settings to make this more user friendly, perhaps.

I've been looking at the branch a bit. there is some experimental total internal reflection work that i've temporarily disabled because its making the result look really dark. I've pushed this to the branch now. It looks like this for me:

Next i'll see if i can address some of the issues with looking straight down through the water. and bring it up to date with latest from master.

[huwb]

Sync'd up now FYI

[huwb]

@mschwarz1 yeah that fix helps a ton and i've integrated it and pushed. Nice!

I made some adjustments to it:

I remove the if condition for camera < 2m from sea level. this broke when going deeper under water, or would break when the wave heights are greater than 2m. EDIT rereading the above you added this for a reason but it doesnt seem to be required anymore on the latest version.. i think!

I made it do a similar job when you're just below the water looking up - in this case it does the opposite, it pushes the skirt geometry away.

Also I added code comments and some links as i could not understand my own hacky code :)

I put a list of some optimisations and the meniscus needs love as it can still get caught in nasty edge cases. I guess just fading it based on view angle would be a good thing to try.

[holdingjason]

@huwb Yep ok great so far so good. I see that keeping the values in check between the two (skirt and ocean) fixes the hard line issue I was seeing before. Now need to play with that via code so that the values I have above water sync back to the original water settings I want vs the matching ones for below water. My guess is you won't really notice the change much as we are still so close to the surface at least that is my hope. Going to do the same things with the postprocessing values as well basically turn on weighting based on distance below water so the effects kick in as you go a little deeper and lessen as you breach the surface. Throwing in a nice water camera effect (running window pane drops for instance) should also help with the transition as well be obscuring things for a second. Once I get it all working will whip up a video and send out the link.

BTW was also going to do a full video of driving the boat around as well, did you ever figure out where you maybe want these videos to go? Kind of see what crest can do section.

Thanks great job as always. I think this is going to look fantastic, not many games support a transition this nice.

[holdingjason]

So how hard would it be to do change how transparency worked a bit so that instead of being liner (I think looking at the code it lerps at a set rate right now) to have it work with various options again similar to how fog works ie can change the rate as well as specify linear, exp, exp2. The reason for this would be to allow very transparent views while near the camera and extended out a ways but then control how quickly it starts to obscure. Again exactly how fog works. Granted it would have to be added not just to the skirt but to the ocean as well but that could be beneficial for the same reasons, near show we could have very clear water up to x depth and then rapidly drop off with more of a exp, exp2 approach.

Talking out my ass here a bit but just wanted to throw out the thought again and see what we would be up against or if there is a better or already existing way to do something like this.

Still need to play with the existing approach to see if I can tweak it to get what I want but just figured I would ask while I do that.

[huwb]

Thanks Jason, A link for now would be great, i haven't settled on a plan here but i might contact you off thread about it.

So i tried to give the meniscus a makeover by applying a texture but even after a few hours i could not get a good result, so i went back to the drawing board and i think i've eliminated the need for it completely. i now get crack free rendering without it:

the trick was to move the verts a bit towards the water horizon, which creates a bit of overlap to avoid cracks. it looks good to me in my tests and was a just a bit of added code.

Having said that the meniscus is still fully supported, and some kind of delineation between above and under might be desirable. I think i'll push a much more subtle, multiply blended version that looks like this for now:

The frozen screenshot does not do it many favours but it looks ok under motion. Hopefully going forwards we could switch it out for an artist authored texture, or perhaps do something fancy like sampling the cube map to emulate how it would work in reality.

[huwb]

So how hard would it be to do change how transparency worked

Just noticed this - not "hard" as in difficult, but would take a bit of dilligence to see if it can be done elegantly (and perhaps it could borrow code from the unity fog stuff?), and not sure if i want to take it on my list right now. happy to discuss it though, but I'd consider this to be a separate issue so perhaps open a new issue if i can provide further assistance.

[holdingjason]

Ok np. I guess I should have started by asking if my guess is correct and this is related to transparency or something else? I can play with it and just see what I can work up. Sure will start another thread perhaps on that.

[huwb]

Oh, both the underwater skirt and the normal ocean surface are refract materials - they sample the scene colours and depth and then apply the depth fog in shader. You can trace the shader variable to see the code - look for _DepthFogDensity . It's exponential depth atm.

[holdingjason]

Oh so it's probably pretty close to what I want now anyway. I can play with it then.

[huwb]

Getting closer to shippable:

• Couple of nice optimisations
• Works in shallow water
• Works for shadowed water
• Made meniscus width tweakable

Don't think its far away now at all, maybe we can get it pushed this week. Testing/feedback welcome

[mschwarz1]

@huwb glad to see my comment was helpful. The clause I had put in about the < 2m or whatever is still needed although only at extremes so you could set it to < 100m or something like that and it would probably still be fine. To reproduce that you can set the camera's y position to -3000 and look down and the skirt will fail. For most people it wouldn't be an issue though. Aside from that also ran into a weird horizon line while just slightly underwater in shallow water.

Will play around with it some more at some point.

[mschwarz1]

Also the meniscus gets weird when looking straight up from underwater. You can probably fix this the same way the underwater skirt failing issue was addressed I would imagine.