Add texture inputs for clouds, moon, starfields to PhysicalSkyMaterial

[Lexpartizan]

Describe the project you are working on:

I'm working on a project https://github.com/Lexpartizan/Godot_sky_shader this is shader sky with day\night cycle.

Describe the problem or limitation you are having in your project:

i write own shader. it is difficult.

Describe the feature / enhancement and how it helps to overcome the problem or limitation:

Physical Sky Material is an implementation of the atmospheric scattering algorithm and allows you to implement physically correct sky colors depending on the position of the Sun, that is, the time of day. That is, it is necessary for the day and night cycle. However, for the sky, one color is not enough, there are present clouds in the sky, and at night there are stars. Therefore, in order for this material to have practical value, you need to add inputs for the textures of clouds, a star field, and possibly the moon(its dificult, becouse panorama). They can be either static or outputs from viewports with other shaders working. To avoid unnecessary texture calls due to performance penalties, you might want to use a checkbox that allows you to use textures.

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams:

cld.rgb *=attenuation; //lighten the clouds depending on the height of the Sun
sky = mix(sky, cld.rgb/(0.0001+cld.a), cld.a); //This code mixing clouds and sky without black borders
// if texture empty, cld.a = 0.0, so the result of the operation will be unchanged, sky.

If this enhancement will not be used often, can it be worked around with a few lines of script?:

To do this on their own, the user needs to be well versed in shader programming. Any novice can specify the texture with clouds. This useful.

Is there a reason why this should be core and not an add-on in the asset library?:

Because the Physical Sky Material is already in Godot. But there's no such thing as an empty sky, so it's less useful right now for novice. You can use it as a Shader for atmospheric scattering as base and add new functions yourself, but this requires qualification. My proposal will significantly expand the possibilities for creating a realistic sky for beginners who are not familiar with shader programming.

[TokisanGames]

Regarding the discussion on texture inputs in the linked discussion, Godot could detect in C++ whether there are textures in the fields and can take this into account when sending the shader to the compiler. The shader does not need checkboxes.

I can see adding texture maps to the procedural sky, but since the PhysicalSky is supposed to be realistic, I wonder if it should move in the direction of having realistically generated moving clouds and other features. Static clouds are only mildly interesting.

[clayjohn]

Ill echo a similar sentiment to @tinmanjuggernaut regarding clouds. How many users will have accurate images of clouds lying around with proper alpha channels? My guess is none. A cloud texture works for your use case because you are generating clouds in a Viewport, but it likely doesn't help others.

As for the night sky, it should actually by quite easy to add, right now I have a comment in the code so users just need to convert to ShaderMaterial and add the texture lookup themselves. We can easily make night sky built in.

Similarly with moon, although a proper moon is more challenging.

In summary, I'm all for night sky and moon. I am not sure about clouds, but maybe we just need to spend more time working out the details. In the long run, I want to make it easy to support a fully dynamic cloud/sky/weather system.

[TokisanGames]

Dynamic, raymarched clouds are available. https://m.youtube.com/watch?v=IQ-yw19xBQ8&feature=youtu.be

https://github.com/danilw/godot-utils-and-other/

They don't respond to light or anything and are a little expensive, but tolerable with a decent card. You can customize their color, speed, thickness. It renders to an Equirectangular texture. The shader originally came from shader toy. He also has a gles 2 version, but I haven't looked at that.

I used the shader extensively in this video. The clouds are at the beginning, but I also use the same technique for the dynamic space background at the end. Though a static night sky texture is better. https://youtu.be/wmgALZzPK3A

Edit: The entire skysphere is raymarched. But we should be able to extract just the clouds.

[Zireael07]

To add to what @tinmanjuggernaut said, I am using a variation of that shader (including reacting to the sun's color) in my racer game on Intel Kaby Lake and the game runs at fairly decent 30 fps. I could not, however, figure out the panorama maths on my own, so I would like to echo the OP's original request for some conversion function.

[Lexpartizan]

@Zireael07

I could not, however, figure out the panorama maths on my own, so I would like to echo the OP's original request for some conversion function.

I completely agree with you. If we can take an example of converting a UV texture to panoramic coordinates in the range -1.1 (in danilw cloud shader), then there is no correct reverse example. Or I can't work with it correctly. As a result, I can't display the moon's texture on the panorama. More precisely, I succeeded, but with limitations. This is one of the main problems.

@tinmanjuggernaut

Dynamic, raymarched clouds are available.

Yes, I also used them in my day and night cycle project. I was able to pull out a few more important variables to adjust the appearance of clouds, for example, the height of the sky and the size of the clouds, this allows you to give a more realistic look, because now with average coverage we have a lot of small clouds. In addition, we would like to note, so as not to forget, that the x-axis is inverted in this Shader when converting UV to panorama. In General, I have slightly improved it.

@clayjohn

I want to make it easy to support a fully dynamic cloud/sky/weather system.

I completely agree that this is what I am trying to do in my project. But I'm not sure whether it's worth overloading Physical Sky Material with this (Shader clouds, for example). I believe the clouds should be in the form of a separate Shader. So that we can use this texture for other purposes. God_rays, for example. I also had this idea, using clouds for shadows. Perhaps something like a transparent sphere, where the transparency will be alpha from the cloud texture. Or perhaps some kind of light Shader. The moon and the night sky can eventually just be used as a static texture. If the Moon doesn't move and the stars can't blink, it still won't be all bad. If you do everything in Physical Sky Material, then you need to add God_rays to it.

How many users will have accurate images of clouds lying around with proper alpha channels?

It is difficult for me to judge, I think such resources are often found in photoshop masters, but I do not know how to work with 2D graphics editors more complex than paint. Because I don't know how to draw and I don't see the point of learning to work in these editors. It is strange that modeling in a blender for some reason turns out well.

[clayjohn]

I don't see a need for panorama conversion math anymore. Sky shaders work in directions now, so there is no need to write a shader that draws to a quad and then convert to panorama coordinates. Just write your logic directly into the sky shader and Godot will handle all the conversions for you.

As for providing a texture for clouds, clouds can be drawn in the subpass buffer, what we could do is just provide access to the subpass buffer from other places so that you can access the cloud texture from other nodes (e.g. for God Ray's and stuff).

[Lexpartizan]

I don't see a need for panorama conversion math anymore

How about moving texturing Moon? I'd also like to think about cloud shadows. No ideas? I'll try using a sphere with transparency this weekend. I don't think this feature is particularly necessary, but what if it's easy to implement?

[clayjohn]

@Lexpartizan Godot doesn't force anyone to use panoramas anymore. The sky shader works in directions. Your primary input is view direction. So there is no converting back and forth. The only exception is when you use an HDRI for the sky, then you need to convert from direction to UV, which is already done for you.

[Lexpartizan]

Your primary input is view direction.

I don't seem to understand how this works. We have the texture of the moon, which is a normal flat image with UV. We need to place it in any direction of the sky without distortion.

[clayjohn]

@Lexpartizan your proposal is for a texture input for the moon. I assumed you want users to be able to assign a texture under "moon" and have the PhysicalSkyMaterial handle drawing it correctly. It has nothing to do with panoramas. At no point does the moon ever need to be converted to panorama or back.

[Zireael07]

@clayjohn: Looks like we're gonna need a dedicated documentation/tutorial page on sky shaders and view direction.

[Lexpartizan]

@clayjohn However, the texture should be displayed on the panorama without distortion. I can only do this with great limitations.

[clayjohn]

@Lexpartizan you won't need to render to a panorama anymore. That's what I'm trying to explain. There is no need to convert to and from a panorama texture anymore in 4.0. So you don't need to worry about the distortion that comes from converting to and from panorama.

@Zireael07 the doc is already up at https://docs.godotengine.org/en/latest/tutorials/shading/shading_reference/sky_shader.html

[Lexpartizan]

I'll try downloading the godot 4 test build and messing around with the sky shaders over the weekend. I think that EYE_DIR is the rd in that cloud shaders from danilw that we all start from. Let's see if we can do something about it.

[ghost]

I am working on a plugin, I was expecting godot 4 alpha to continue working on it, but I can release a version for 3.2 if they are interested.

https://www.youtube.com/watch?v=agMQn0h5Vyo

[Lexpartizan]

@Ekho7 It would be interesting to look at the code, there are several features that I would like to adopt.

[clayjohn]

I tried implementing a starfield texture and moon texture, they were easier than I though. This weekend I will polish it up and make it easier to use.

Right now the PhysicalSkyMaterial uses the first DirectionalLight as a sun. What do you think about the second DirectionalLight being used for the moon? That way the size, direction, and intensity of the moon can be controlled with a DirectionalLight.

[TokisanGames]

Does this have 4 DLs like procedural sky? Since a sun or a moon are both sky lights, they could be multiple moons or multiple suns. The only difference is if they have a lower power and a texture.

[Lexpartizan]

Right now the PhysicalSkyMaterial uses the first DirectionalLight as a sun. What do you think about the second DirectionalLight being used for the moon?

This good idea. I use one lightsource for moon and sun and just switch light position, depending on sun height. This imposes some limitations, but I wanted to use a minimum of light sources to increase performance. This works because I have the Moon and the Sun opposite each other and the moment of switching is not noticeable due to the lack of shadows, since the Sun is already shining from below.

In addition, the same light source is used as a lightning (storm), so that the shadows are from lightning. I just temporarily move it to the lightning position.

multiple moons or multiple suns.

Algorithm atmospheric scaterring only works for one sun, it seems to me. But you are right, do not get stuck in the sky of the Earth.

[clayjohn]

Does this have 4 DLs like procedural sky?

No because the math is based off of measurements taken of the earth. The algorithm doesn't support more than one sun. The intention is to be physically-based (hence the name).

[Lexpartizan]

Such a stupid question, I didn't notice before that my sky Shader doesn't make reflections. I thought it was insignificant. However, now all metal surfaces look black. If there is any way to easily solve this problem? I also want to add that the cloud texture should use (-UV) when overlaying the panorama. Only in this case it works normally with god_rays. is see why it is inverted by y (flip-y checkbox), then we have not understanding problems with x.

How about immediately optionally implementing the god_rays Shader in physical sky material, since it uses clouds, and light source sun, and also refers to the sky.

[Lexpartizan]

I looked a little at the physical sky material Shader. while not delving into the topic, the first glance. What liked:

1. The very lighting is working correctly and the color pink in the sunset. I do this manually in the script. It's the same with the brightness of the lighting. I do not understand how this works, because the brightness of the light source and its color do not change. But this cool.
2. A Shader written in a professional manner, there is no logical branching and, most importantly, there are no cycles. The Shader must be fast.

What not liked:

1. The sun does not have a pronounced shape, sometimes it is too dim. Since there used to be a fairly pronounced circle, I think this is easily corrected.
2. I was not able to make significant changes to the image using the settings. In particular, the color of the sunset was unchanged. Only its intensity varied.
3. It is not clear with the performance. When I move something, I have small friezes. But I think this is due more to the dampness of the Godot 4.0. I have the entire console spammed ERROR: Condition "!windows.has(p_window)" is true. at: window_resize (drivers/vulkan/vulkan_context.cpp:723)

[Calinou]

The sun does not have a pronounced shape, sometimes it is too dim. Since there used to be a fairly pronounced circle, I think this is easily corrected.

The sun's appearance depends on the Angular Size property in DirectionalLight. Try increasing it above 0.

We should probably have a separate property for this, as you may want to have a sharper-looking sun without the cost of PCSS-like shadow maps.

It is not clear with the performance. When I move something, I have small friezes. But I think this is due more to the dampness of the Godot 4.0.

This is now fixed out of the box thanks to https://github.com/godotengine/godot/pull/40272. (Previously, you could have changed the sky update mode to Real Time in the Environment resource.)

[Lexpartizan]

Thanks!

[Calinou]

@clayjohn What do you think about this:

We should probably have a separate property for [environment sunlight size], as you may want to have a sharper-looking sun without the cost of PCSS-like shadow maps.

[clayjohn]

We should probably have a separate property for this, as you may want to have a sharper-looking sun without the cost of PCSS-like shadow maps

@Calinou, I struggled with this when I added the sun size property to the shaders. In the end, I felt that it would be better to just let users alter the shader if they wanted to control the sun size manually.

Im not sure what a better setup would look like. Right now light size affects 3 things:

1. PBR shading
2. PCSS shadows
3. Sun size in sky shaders

The concern is that you may want visually larger lights without incurring the cost of PCSS. So maybe a proper solution is to allow users to increase light size without enabling PCSS. So a "force_disable_pcss" option for low-end devices.