Setup basic rendering pipeline

[LukasKalbertodt]

(this issue contains a big chunk of explanation for readers that might be unfamiliar with the topic)

While we have a mathematical "viewing pipeline" as well as an OpenGL "graphics pipeline", the word "pipeline" is also used to describe the steps in a complex rendering scenario. You'll hardly find any game that writes the final color for each pixel to the screen directly. It's rather the case that there are multiple steps each of which draws to a texture/an image which is not displayed on the screen (off-screen buffer). This is especially important for post processing effects (one easy example is color grading).

In OpenGL "things to draw to" are called frame buffers. Each step in the rendering pipeline has one or more frame buffers attached to it, to which the step will draw. The final step always draws to the default frame buffer (the screen), which is called Frame in glium.

The most important task of this issue is to implement a basic structure for rendering HDR images. High Dynamic Range means that each pixel has a large set of possible values, in contrast to the tiny 0 to 255 range we can use on our screen. Since illumination values vary a lot in the real world, we have to be able to save HDR values to do realistic rendering.

It's actually not too hard to get the HDR rendering started: instead of using 8 bits per color channel we will use 16 or even 32 bits (f32) per channel. The hard part comes later: "tone mapping" describes the process of compressing this huge float range back into [0, 255] to represent it on our screens. But doing proper tone mapping is another task -- this issue is only concerned with the setup of the basic pipeline.

In order to do that you have to:

• create a frame buffer with a fitting HDR format
• draw the whole world into that buffer
• add a new rendering step which draws this frame buffer texture on the screen again while compressing the color range

The last step which will contain the tone mapping should be very simple in the beginning. For example the HDR color could just be clamped into the 0...255 range (this is pretty much what happens without HDR).

Later on, there can be many many more post processing effects:

• bloom and lens flare: both are somewhat related to HDR rendering/tone mapping. Bloom describes the effect that very bright regions of an image "leak" the light into neighbor pixels. Due to imperfections in physical lenses and camera sensors, lens flare may occur in real world videos. This only describes the effect that bright spots sometimes cause other pixels to be brighter as well.
• image smoothing: instead of using real multisampling, post smoothing algorithms are used. The purpose of the smoothing is to get rid ugly pixel-ly edges. There are some rather interesting algorithms, one of which is FXAA.
• color grading: changes the temperature of hue of the resulting image. This is often done in film to create a specific athmosphere.
• screen space occlusion: one very interesting algorithm is SSAO (screen space ambient occlusion). This algorithm simulates indirect lighting by online looking at the depth values of the resulting image. It's a very rough approximation but can give nice results with some fine tuning

Especially the last algorithm is something that we can implement in this practical. As already described, SSAO needs the depth of each pixel; sadly it's not yet possible to access the default depth buffer of the GPU via glium. Therefore we will create our own depth buffer (additionally to the GPU provided one) to save the depth value for each pixel. This means that we want to use multiple frame buffers at once (one for color, one for depth).

Here are some important resources:

• Surface trait of glium
• framebuffer module of glium

This is a rather complex topic, so remember to implement this step by step, with preferably very small steps :wink: I'd even suggest trying a few things in a test project (read: not in this project at all). The examples/ folder of glium could help a lot. Try to find an example that looks like it might help and play with it until you got a basic feeling for the topic. Then you can talk to me about how to do it in our project.

This issue can be subdivided also: one subgroup could think about the general structure (HDR) while the other one already thinks about either FXAA or SSAO.

I hope you can live with not contributing to the project directly but rather doing research first. If you have any questions, let me know, but remember: a big chunk of this issue is to find the necessary information and to also try out on your own.

Please also excuse any bugs in my language, it's already late enough :frowning:

[LukasKalbertodt]

As promised, here are some links with information about rendering pipelines from commercial games:

• Deus Ex Graphics Study
• GTA V Graphics Study

There is the small risk that all this information is overwhelming and will rather confuse than help you. But maybe it's a nice read to get an idea about how stuff is done.

[Cranc]

article i found helpfull concerning HDR implementation

https://imdoingitwrong.wordpress.com/tag/hdr/

[Cranc]

does plantex enable the openGl depth test ? by default openGl does not enable it.

[jonas-schievink]

@Cranc Yes, it's done before (I think) every draw call using DrawParameters

[LukasKalbertodt]

This is kinda done...