Dynamic, safe and plugin-able rendering system

[a-teammate]

New rendering system

Disclaimer: I know about #101 and #310, but this not only about tesseract and not only about lights

We want a robust, easy-to-use system which allows dynamic worlds. Currently we have to prebake lightmaps, which is a time-consuming process of the editing experience.

There is another Cube2 fork called "Tesseract", which replaces lightmapping with deferred rendering. That change reduces the time to create lights significantly (only step required to do manually in Tesseract is calculating normals), as lights are rendered in realtime. As Tesseract is built in the same spirit as Sauerbraten, it's using various techniques to allow high framerates. It's not state of the art anymore, but close enough. However as it's a very close relative of Sauerbraten, it's source is also written in a similar manner.

Issues:

• Code is not drop-in-able
  • not modular, not sticking to our don't reinvent the wheel policy
• Graphics code manually optimized for various graphic card vendors
  • process which needs a lot of expertise (and time)
• Shader code written in cubescript
  • a dead language, see our docs
  • same for the asset pipeline of course

The good thing is that the Tesseract code has a lot of comments (compared to the Sauerbraten code base). I suggest we design the new system as follows:

BGFX as backend

• abstracting away any vendor specifics
  • or graphics API specifics (OpenGL, Vulkan, Directx, Mantle..)
• a state-less API
  • statefullness is the source of most errors for beginners as well as experienced users of OpenGL
• a modular shader system
  • cons: also using a selfwritten language
  • we should help them make the language even more GLSL alike
• prebuilding shaders + building shaders in another process
• additional performance enhancing optimizations
  • minimizes state changes
  • that state sorting also allows to commit render passes at the same time => possibly cleaner code
  • Con: How well tested is it for complete high-performance engines?

Tesseract on top

It offers:

• deferred lighting
  • gbuffer + shadowmaps (including the shadow atlas solution)
  • cascaded and reflective shadow maps
  • tilebased shading
    • a culling method for lights (/particles)
    • newer method: clustered shading
• Different passes:
  • decal pass
  • transparency pass
    • cpu based sorting, can be replaced with BGFX
    • refraction shader
  • sky? tess has an additonal sunpass
• reflections? how are they handled?
  • reflective shadow map, everything else purely in shaders
  • alternative: stochastic screen space reflections
• normals generation
• timers
  • redo with a library or stdlib
  • find way to do GPU timers with bgfx
• screen "management"
  • half res rendering (setupscale)
    • has: linear, cubic upsampling
    • add: bilateral upsampling (add depth tex to bilateral shader)
    • add: adaptive upsampling: vary resolution to stabilize frame times
  • add: frame manager
  • multi resolution, second viewport ("splitscreen"), Intermediate frames
• HDR
  • HDR framebuffers are ofc mainly OGL code -> replace
  • tonemap shader
  • bloom
    • otherwise you won't notice a thing
• fog shader
  • replace with generic new system: map postfx shader
  • fog variants: sideeffect culling also needs replacement
  • or maybe dependency injection for shaders? -> shader variants
  • or fake di in bgfx shaderc: different shaders for that, see semi-dynamic plugins
• global illumination
  • method used: radicance hints

things we'll drop:

• grass rendering
  • it sucks in its current form, maybe add another way later on
• msaa (probably)
  • its a very easy method (but there exist better ones)
    • is there another need for msaa methods besides antialiasing? maybe upsampling?
  • add a screen-space method instead
    • and maybe a sub-texel method (msaa is also working with subtexel precision)
    • Tesseract offers a bunch of ss methods thankfully - newer ones exist too though

ToDo

THIS WON'T HAPPEN TOO SOON!! (if I do this) as other things have prio: InexorTree and the sourcebase modularization and cleanup

Tesseract code is in octarender, renderlights, rendergl and material + envmap generation from texture.cpp Loose ends can be decoupled using inexors better modularized sourcebase, bgfx (and the InexorTree for the new managers).

• [ ] create bgfx conan package
• [ ] add simple bgfx testscene
  • needs texture loading + shader loading + some triangles
• [ ] add tess timers
• [ ] add tess gbuffer to testscene -> render gbuffer
• [ ] add tess normal generation for cubes (necessary here?)
• [ ] replace testscene triangles with cubes
• [ ] add sky pass
• [ ] add tess lights (including culling method) to testscene
• [ ] add tess shadowmaps to testscene
• [ ] add tess reflective shadowmaps to testscene
  • includes envmap generation?
  • Optional?
• [ ] add tess decals to testscene (without texmanager changes: but a simple copy)
• [ ] add tess transparency "pass" to testscene
• [ ] add tess HDR rendering to testscene (including bloom shader)
• [ ] add tess GI "pass" to testscene (if radiancehints outdated: do that later, better)
• [ ] find Shader plugins solution
• [ ] replace inexor octarendering with testscene rendering
  • [ ] add new ShaderManager
  • [ ] modify texture and models manager
  • [ ] modify model textures + shaders
  • [ ] reflections
  • [ ] add tess materials
  • leave fog code out for now
  • leave minimap and modelpreview rendering out for now
  • maybe let testscene rendering stay afterwards?
• [ ] add ScreenManager
• [ ] add minimap and modelpreview rendering
  • find a generic system? SceneManager? or modularize: functions are too long

---

shader plug-ins:

• static plugins: plugin-folder autoexecuted in one pass
  • i.e. usually one can combine multiple passes with same input to one: e.g. one shader for gbuffer creation (which outputs into multiple buffers i.e. normal, depth, .. ) But we could combine the screenspace passes as well (light/GI/AO..) as they share the same input (the gbuffer) Best would it be however to bundle them loosely: just drop-in a shaderfile in a folder and it will get executed as well BUT: is this cache-unfriendly for the GPU? noticeably and on what hardware? Otherwise keep multiple passes or automatically tile into multiple passes -> does bgfx already do this?
• "semi-dynamic" plugins like variant generation
  • e.g. a fog or water variant could be generated from a "VARIANT(outputbuffer = functioncall(inputdata);)"
    • implement in bgfx Shaderc?
    • or is there macrolike preprocessing in shaderc which is usable?

Possible follow up issues

• add masterserver benchmark feedback system
  • client runs benchmarks, collects stats, add system info, sends stuff to masterserver, masterserver saves to db, dev makes diagrams, dev understands bottlenecks
• readd fog with generic system (see 'Questions open/shader plug-ins')
• readd antialiasing screenspace method
• add antialiasing texellevel method
• add ambient occlusion algorithm
  • or find one ourselves? idea: stochastic method + bidirectional reprojection (could be applied to reflections as well)
• add ScreenManager->FrameManager
• add "ScreenManager->AdaptiveBilateralUpsampling"-functionality
• add "ScreenManager->IntermediateFrames"-functionality
• replace Tiled shading with clustered shading
• add Splitscreen: more than one ScreenManagers active the same time: multi scenemanager?

Furthermore for a "you map it, it's instantly production ready"-like experience, we will need a solution for:

• the view cell calculation (Sauerbratens clipping method)
  • maybe a LoD(Level of Detail) system + some dynamic clipping
• the normal generation and remipping
  • maybe just straight-forwardly as you map it, or in a background thread

---

Other:

• PBR
  • Physically based rendering: allows predictable (because realistic) lighting in all environments
  • in Tesseract the decision was made against this direction
  • in particular a correct lighting model (linear model) was tried initially
  • but it looked shitty imo (or maybe just unfamiliar?)
  • -> maybe go all the way before judgement?
  • the revamped Ogre 2.1 engine offers (probably good) BRDFs for PBR
  • needs texture revamps + loader changes + new shaders

@bkaradzic I just highlight you here to let you know :)

[IAmNotHanni]

Wow what a ticket :)

[bkaradzic]

👍

cons: also using a selfwritten language

Based on GLSL, and most of existing GLSL code trivially ported (close to copy&paste) to it.

Con: How well tested is it for complete high-performance engines?

Let me know if you have any perf issues with it.

Also feel free to ask any questions https://gitter.im/bkaradzic/bgfx or open issues in main repo.

[Croydon]

things we'll drop: grass rendering it sucks in its current form, maybe add another way later on

It is fine for me if it would cost enormous amount of time to keep it with all of these changes, if this shouldn't be the case then please keep it. Grass rendering is a rather basic feature we should keep, even if it might not be the best it is better than not having it all.

Rest sounds good to me :)