Global cloud rendering

[pjcozzi]

@immiao @byumjin this would be a great real-time rendering project for the winter/spring!

Phase 1 - billboards

• [ ] Procedural billboard texture generation from various cloud types
• [ ] Navie rendering with Cesium's billboard engine
• [ ] Post processing
• [ ] LOD rendering if needed

Phase 2 - Volumetric clouds

• [ ] Procedural generation from noise, etc.
• [ ] GPU ray marched rendering with secondary ray effects

Later

• [ ] Connect to real-time weather data

Resources

Inspiration from GeoFS

• https://twitter.com/GeoFlightSim/status/913410574146572289
• https://twitter.com/GeoFlightSim/status/918851600348844032
• https://twitter.com/GeoFlightSim/status/916324430476972033

@ron-popov and @yotam180 may have input.

Cloud rendering

• Volumetric Clouds. 2016
• Real-Time Cloud Rendering. Eurographics 2001. Early and influential work using billboards
• Nubis: Authoring Real-Time Volumetric Cloudscapes with the Decima Engine. SIGGRAPH 2017
• The Real-Time Volumetric Cloudscapes of Horizon Zero Dawn. SIGGRAPH 2015. @MegSesh may have feedback on this approach
• Modeling, Lighting, and Rendering Techniques for Volumetric Clouds. Game Engine Gems 2
• Lots of papers and apps at vterrain.org
• Resources from the twitterverse
• Dissecting “Tiny Clouds”

Cesium rendering engine

• Rendering a Frame
• Renderer Design
• The Graphics Stack

[xtassin]

One link I find interesting (billboard based method) is dead on the Vterrain page. Here is a the paper http://twvideo01.ubm-us.net/o1/vault/gdc04/slides/realistic_and_fast_cloud.pdf

How it's done in GeoFS:

• uses billboards with a variable density based on distance from camera.
• Billboards are destroyed and generated as the camera moves
• Flat clouds are 3D models (quads) with a transparent texture
• This works only when camera is close to the ground. Will look strange from high/orbital altitude.
• has to fix billboards rotation at each frame when camera roll changes (something is wrong with Cesium's billboard implementation when it comes to "up" orientation)
• entity color is modified according to a value ramp based on time of day for dusk/dawn effect.

[pjcozzi]

@xtassin awesome, thanks for the summary!

[MegSesh]

Hi all,

In regards to the Horizon Zero Dawn Volumetric Cloudscapes presentation https://www.guerrilla-games.com/read/the-real-time-volumetric-cloudscapes-of-horizon-zero-dawn, I worked on implementing this as a C++ plugin in Nuke during my Pixar internship this past summer. A couple other GPU current students and I (Aman Sachan, Joe Klinger, and Dan McCann) are planning to implement this for our final project this semester as well.

The 2015 presentation offers great insight on the approach overall, as well as the motivations behind it. It essentially breaks down into the following sections:

• Goals / Explorations / Method comparisons
• Modeling
  • Raymarching through multiple low and high resolution 3D textures created from a combination of low and high resolution Perlin and Worley noises
  • Describing the weather authoring system they used to dictate cloud coverage, type, and precipitation
    • This and their density model is described much much better in their Nubis 2017 presentation http://www.advances.realtimerendering.com/s2017/index.html
  • Lighting
  • Achieving multiple-scattering using a combination of Beer's Law and the Henyey-Greenstein function
  • Using a combination of beer's law and other equations to create a more "powder" looking effect on the clouds
  • More thorough description in their Nubis 2017 presentation http://www.advances.realtimerendering.com/s2017/index.html
• Rendering
  • Describing the scale of the world, placement of the clouds in the atmosphere, and their raymarching methods through the textures and to the light
• Optimizations
  • Some brief mentions of ways they optimized. Better described in the GPU Pro 7 book (below).
  • More thorough explanation in their Nubis 2017 presentation http://www.advances.realtimerendering.com/s2017/index.html

The GPU Pro 7 book https://www.amazon.com/GPU-Pro-Advanced-Rendering-Techniques/dp/149874253X contains a more descriptive article based on this presentation and goes into a little more implementation detail. So this book would be another great resource.

Let me know if you have any other questions! -Meghana

On Fri, Nov 3, 2017 at 8:04 AM, Patrick Cozzi notifications@github.com wrote:

@xtassin https://github.com/xtassin awesome, thanks for the summary!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/AnalyticalGraphicsInc/cesium/issues/5962#issuecomment-341685495, or mute the thread https://github.com/notifications/unsubscribe-auth/AFKijlJF704bD5msfaNEPZVTcCC0_CAOks5sywFfgaJpZM4QP2ss .

-- Meghana Seshadri M.S.E., Computer Graphics and Game Technology University of Pennsylvania School of Engineering and Applied Science meghana.seshadri@gmail.com

[pjcozzi]

@MegSesh great summary, thanks so much for the notes! @immiao, @byumjin, and I will stay in touch as this moves forward.

Great final project idea as well!

[pjcozzi]

CC #797 - some WebGL 2 features like 3D textures will be required - we can have a fallback for WebGL 1.

[pjcozzi]

Forum discussion: https://groups.google.com/forum/#!topic/cesium-dev/Hpzap5oNKJg

[pjcozzi]

Repo for weather API integration by @immiao: https://github.com/immiao/cesium-weather

[pjcozzi]

@immiao could you please evaluate these weather APIs:

• https://aws.amazon.com/public-datasets/nexrad/
• https://openweathermap.org/api
• https://www.wunderground.com/weather/api/
• http://customweather.com/products/
• https://spire.com/data/weather/
• http://apidev.accuweather.com/developers/

[immiao]

Sure!

[pjcozzi]

@immiao NASA is also a great source of weather data, please evaluate:

• NASA Global Imagery Browse Services (GIBS)
  • Available layers
  • API
  • Cesium demos
• Forecast modeling
  • https://gmao.gsfc.nasa.gov/weather_prediction/
  • https://gmao.gsfc.nasa.gov/reanalysis/

[pjcozzi]

Related killer projects by CIS 565 students

• Real-time Cloudscape Rendering in Vulkan by @Aman-Sachan-asach and @MegSesh
• Marshmallow - Vulkan-based implementation of clouds from Decima Engine by @klingerj and @mccannd

[AmanSachan1]

Our project has been renamed to Meteoros https://github.com/Aman-Sachan-asach/Meteoros.

Best, Aman Sachan

On Tue, 5 Dec 2017 at 09:15 Patrick Cozzi notifications@github.com wrote:

Related killer projects by CIS 565 students

• Real-time Cloudscape Rendering in Vulkan https://github.com/Aman-Sachan-asach/Vulkan_Cloudscape_Rendering by @Aman-Sachan-asach https://github.com/aman-sachan-asach and @MegSesh https://github.com/megsesh
• Marshmallow - Vulkan-based implementation of clouds from Decima Engine https://github.com/mccannd/Project-Marshmallow by @klingerj https://github.com/klingerj and @mccannd https://github.com/mccannd

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/AnalyticalGraphicsInc/cesium/issues/5962#issuecomment-349316244, or mute the thread https://github.com/notifications/unsubscribe-auth/AHhfdr0c_J1owMNIrRW0qnLqwRHf9Pd3ks5s9U__gaJpZM4QP2ss .

[pjcozzi]

• Also see Volumetric Clouds based on "Volumetric Clouds and Mega- Particles"

[pjcozzi]

• Perhaps another datasource to check out: EarthServer: 1+ Petabyte Analysis-Ready Datacubes

[likangning93]

Not sure if this'll be relevant, but it's cool and well written, Mapbox blogpost on wind sim: https://blog.mapbox.com/how-i-built-a-wind-map-with-webgl-b63022b5537f

[likangning93]

some WebGL 2 features like 3D textures will be required - we can have a fallback for WebGL 1.

Reading through the Horizon Zero Dawn presentation, theoretically couldn't we pack the 3D texture to 2D? Back-of-the-envelope math, 128^3 takes much less GPU memory than 2048^2. Would it be too slow though? [EDIT] b/c maybe there's trilinear interpolation in hardware or something for Webgl 2 3D textures...

[pjcozzi]

Packing into a 2D texture could be fine. For example, see "Real-Time Volumetric Lighting for WebGL" - https://github.com/WebGLInsights/WebGLInsights.github.io/releases/download/v1.0/WebGL.Insights.-.Patrick.Cozzi.pdf

[AmanSachan1]

Just as a note, the 3D textures define the cloud using all the input parameters that are provided by the the weather map. The weather map described in the paper was a 512*512 texture that defined a 100,000sq km. Or the weather map can be replaced by a simulation. So over all if you pack the cloud textures (low frequency 3D texture, high frequency 3D texture, and a 2D curl noise texture), it should not be that terrible in terms of memory. Also, apparently the original authors weren't able to perfect it to the point that they could ship it with the game.

On Mon 18 Dec, 2017, 12:53 PM Patrick Cozzi, notifications@github.com wrote:

Packing into a 2D texture could be fine. For example, see "Real-Time Volumetric Lighting for WebGL" - https://github.com/WebGLInsights/WebGLInsights.github.io/releases/download/v1.0/WebGL.Insights.-.Patrick.Cozzi.pdf

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/AnalyticalGraphicsInc/cesium/issues/5962#issuecomment-352504091, or mute the thread https://github.com/notifications/unsubscribe-auth/AHhfdp8d-hHle3dWHPB3LU4bMzyZ0f6Tks5tBqaHgaJpZM4QP2ss .

[pjcozzi]

• SIGGRAPH 2016 course: Physically Based Sky, Atmosphere & Cloud Rendering