deck.gl port

[farfromrefug]

@OmarShehata by any chance would you have any knowledge in deck.gl ? I am struggling to port your outline pass to deck.gl They have an effects/pass system which should allow me to get the normalBuffer but i am not seeing how to do it :s Thought it was worth asking. Thanks

[OmarShehata]

Hey @farfromrefug , I have worked a bit with Deck.gl. Can you push what you have so far to a repo/codesandbox/Glitch etc? I can try and take a look sometime this week.

[farfromrefug]

@OmarShehata first thanks a lot for answering! I set up a repo here https://github.com/farfromrefug/3dmap_webgl with what i have so far. So after looking a lot at the deck.gl codes and at your three js version i felt like "copying" the deck.gl pass for shadow was the right way to start. It was using a depthBuffer which was necessary for my port. So i kind of put all in place but then i am pretty much stuck:

• dont know how to setup the normalBuffer
• feels like i have what i want in outline.js but not sure how to have the readDepth glsl function working. Should i use a projection matrix like it is done in the shadow pass here

I am kind of new to all that webgl stuff and i am learning as i go :D Hope i did enough for you to take a look at it.

Again thanks a lot for even looking at this!

[OmarShehata]

Thanks for putting your code up @farfromrefug ! I wasn't able to run it because it looks like I need some local elevation data (http://localhost:8080/data/BDALTIV2_75M_rvb) ? I just see a white screen with lines & labels.

But seeing this helps. I think porting to Deck.gl will be a bit tricky. The normal buffer is the hardest part of this effect because it's not a standard thing engines typically support. To do this you'll need a way to render to texture (looks like Deck.gl doesn't officially support this? https://github.com/visgl/deck.gl/issues/3313) You need this because you need to re-render the scene, but instead of using a default material, you'll create a custom material that colors the terrain using the normals.

If the engine doesn't support custom materials, that step will be tricky too. This isn't supported directly by CesiumJS either, but I have a forum post here describing how to change the terrain shader to render the normals (https://community.cesium.com/t/forcing-a-low-poly-look-on-cesium3dtileset/9344/4?u=omar). So if you were doing it in CesiumJS you'd create a uniform boolean that is passed to that shader, so you render the scene once normally, then once with the normals, and save that to the texture, and that's your normal buffer.

For what it's worth, in CesiumJS the globe materials here are the closest thing that combines all the pieces you need: https://sandcastle.cesium.com/?src=Globe%20Materials.html. These effects are created as shaders that have access to the terrain's normal and aspect. So the easiest route there would be modifying one of those built-in shaders.

[OmarShehata]

I would definitely love to see more geospatial engines support creating effects like this more easily, have you posted your question in the Unfolded community (https://unfolded.page.link/slack-signup) ? Perhaps that'll help generate some interest to support some of these fundamental pieces (render to texture, custom materials) similar to how PlayCanvas and ThreeJS do.

[farfromrefug]

@OmarShehata First thanks a lot for looking into this and taking the time to answer! I also would love to be able to do that in all the geo libs around. recently i also discovered iTowns which is pretty good too. Now in my use case i need a very "slim" one because i run it in a special environment (kind of like React Native with direct bindings from JS to OpenGL, except it is not ReactNative but Nativescript :))

• Three.js: amazingly slim but quite hard to get the all geo spatial thing running smoothly
• deck.gl: Pretty amazing result but missing oultine support(not possible yet) like you explained
• cesium: A bit bigger and still missing outline support, though as you explained it would be easier to implement. I am also not sure about rendering the labels like i do (on screen see screenshot below).
• itown: much bigger. Post processing seems quite easy. not sure about normals yet.

I am on the deck.gl slack. Will ask there pointing to this discussion as i find it very interesting and self explanatory.

https://user-images.githubusercontent.com/655344/116363136-1bd14580-a803-11eb-9a3e-6f2273206e13.mp4

[OmarShehata]

itown: much bigger. Post processing seems quite easy. not sure about normals yet.

iTowns is ThreeJS based so this should be the easiest one to get working! In fact it should "just work" with the current code if it's using the latest version of ThreeJS. I would:

• Look for any iTowns examples using EffectComposer, which will allow you to render all the different passes you need for this effect, like here: https://github.com/OmarShehata/webgl-outlines/blob/9a7d67298346a64252ba2c1c0be89400316cecbb/threejs/src/index.js#L45
• Otherwise, you may need to look for something like renderer.render( scene, camera ); if that's handled internally by iTowns, you'll need to override that (because that needs to be replaced by the effect composer, which handles rendering all the layers)
• If you can get a reference to the ThreeJS scene in iTowns, you can set the overrideMaterial, which is how this effect draws the normal buffer (it renders the scene with everything using this special material): https://github.com/OmarShehata/webgl-outlines/blob/9a7d67298346a64252ba2c1c0be89400316cecbb/threejs/src/CustomOutlinePass.js#L61

Best of luck! I'd be very interested if you're able to get this working with any of the libraries, if you're able to share the technique/code publicly. I think it'd be very useful for others.

[farfromrefug]

@OmarShehata yes i will try to get it to work for as many libraries as possible. I ll make PRs here if i get it. also today i discovered this https://github.com/tentone/geo-three And it is pretty cool for my need. Very slim and also has a very nice feature: height computed through GPU! I need to try and port your code so that it work with it.

[OmarShehata]

geo-three looks very cool, thanks for sharing! I hadn't encountered it before.

[farfromrefug]

@OmarShehata kind of got what i want with geo-three I might need your help on something i want to change. In the gif below we can see that "terrain" get accentuated as you further a way from them. I would like the contrary. The reason is that if you look very far away it almost all white and hard to "understand" the terrain. I am thinking that if i manage to inverse the effect i would get a better result. I tried to play with multiplierParameters but i could not find where i could inverse the effect. Do you have any idea if this would be possible? not sure you even get what i mean...

[OmarShehata]

@farfromrefug I get what you mean. I think what's happening here is that differences in depth further away from the camera are smaller, due to less precision. So if the shader says "color anything white as long as the depth difference between it and the closest pixel is bigger than 0.1", that may represent a smaller distance close to camera than further.

If that is the case (and I would corroborate this by confirming this effect is coming from the depth diff. Try setting normalMultiplier to 0. If you still see the same behavior then it is indeed the depth issue), then the real solution is a reverse depth buffer or logarithmic depth buffer (see https://gamedev.stackexchange.com/questions/117995/why-does-reverse-depth-buffering-provide-more-precision-with-dxgi-format-d24-u and https://outerra.blogspot.com/2009/08/logarithmic-z-buffer.html) or multiple frustums, both of which are more of an engine change.

The easier option is just instead of using a constant:

"color anything white as long as the depth difference between it and the closest pixel is bigger than 0.1"

Tweak that value based on how far away this pixel is from the camera (which is the depth value itself)

"color anything white as long as the depth difference between it and the closest pixel is bigger than 0.1 x current_depth"

The other alternative is a geometry-based outlines algorithm, which would look the same regardless of distance (as opposed to this screen-spaced based one), which I've been wanting to find the time to look into.

[farfromrefug]

@OmarShehata Thank you so much for your insight on all of this . It is pretty awesome and it is what i figured. I also that the same thing happens with normals diffs and actually seem to have a bigger effect. I tweaked the multipliers here and ended up with this for now which i am already happy with (looks better when you open the browser dev settings and simulate a mobile phone) https://farfromrefug.github.io/geo-three/examples/

I already tried your idea of taking into account the depth to color the value. But i had a side effect of "losing" major outlines (between the terrain and the sky) at distance. Need to push this further Thank you!

[OmarShehata]

Very cool, thanks for sharing your progress here! It definitely makes me curious about exploring other techniques that may be more suitable for the geo context. I know Mapbox for example has a layer that bakes in these lines one way to generate lines that are consistent and look good (do it offline) https://docs.mapbox.com/studio-manual/examples/hillshade/

[farfromrefug]

@OmarShehata Yes i already use that and draw hillshading using terrain rgb tiles :D I also do slopes using that technique. And now thanks to you i can do 3d maps :D No what i found out today is that depth has almost no weight compared to normals. If i completely disable the depth and only look at normal i get almost the same result ! I need to push it further but what is really cool about that is that i can make the post process much faster by removing the need for the depth buffer but also the normal target! In my case as i draw the normals directly with the terrain material shader (as you do here) i can simply use the sceneColorBuffer as the normalBuffer. The perf improvement is huge (especially on mobile). This is what i get with only the normals.