Extract Rendering Setup From Camera

[ilexp]

Summary

Right now, every Camera instance has its own configuration of rendering passes. The only viable way to share them among different Cameras is using a Prefab, which may have side effects that are not intended or wanted by the user. Sharing rendering setups is the default case, so it should be easy and straight-forward.

Workaround

• Create a Camera Prefab and use it to share the rendering setup.

  Analysis

• A new Resource type could be created to store the rendering setup information, with each Camera referencing it.
• All Camera-related information in a rendering pass should be expressed as a relative value, so it scales according to each using camera's settings.
• When no Resource is referenced, a default setup should be used. Also introduce the default rendering setup to the available default resources.
• For cases where a non-shared / unique / custom rendering setup is used, it shouldn't be too bad to just create another rendering setup resource.

[ilexp]

Issue A: Temporary / Programmatic Passes

• There are cases where it is required to temporarily add or remove passes from a specific camera.
• This is done by the editor in order to render its custom CamView overlays.

Solution A1

• A camera could copy all rendering setup passes and provide a pass list locally as before.
• This does not work well with live updates in editor and core when the rendering setup is modified. A direct reference is preferred.

Solution A2

• Provide a camera interface to add and remove additional passes programmatically.
• The rendering setup resource is treated as opaque and cannot be directly modified per-Camera.
• It should be possible to insert a custom pass before or after any pre-defined pass, or at the beginning / at the end. A pass Id field can be used to identify these relative pivots.
• The rendering setup is neither copied nor modified. Instead, the camera maintains an internal list of all applied passes that is not exposed and updated each frame right before rendering.

Issue B: Per-Camera Pass Events

• Each rendering pass has some events that allow external sources to tie in and add some custom draw device operations.
• This does not mix well with rendering setup being a "static" Resource, since those events are used for per-Camera operations.

Solution B1

• Move these events from rendering pass to the camera component and add an additional parameter that specifies which pass is being dealt with in the event.
• Also, provide an Id field to each pass, so they can be identified without relying on direct references.

[ilexp]

Note: Consider renaming RenderPass to RenderStep, since pass is a term that is usually used in the context of rendering an object in multiple stages.

[ilexp]

Note: Consider going one step further and make the rendering setup itself (by default, it's "foreach Camera") configurable.

• It would allow to easily do post-all and pre-all effects when multiple Cameras are present.
• It would make the old "Camera predicate" hack obsolete, because the rendering setup could filter which Cameras to render when desired.
• It could either be a clear distinction between "Camera Setup" and "Rendering Setup" Resources...
• ...or a unified "Rendering Setup" resource with both pipeline setup and camera setup, where the pipeline setup would only be used when the Resource is set as the primary / default setup in application data. When a camera uses a specific non-default rendering setup, only the camera setup part would have an effect.
  • Might be easier / less convoluted and wouldn't have the problem introducing an almost unused Resource type just for the rendering setup.

[ilexp]

Note: Consider re-introducing Camera viewports, so each Camera can define the (relative) region to render to. In addition, each rendering step should allow to specify a viewport rect relative to the Camera viewport.

[ilexp]

Note: Consider allowing a Camera to specify a base rendertarget that will be used whenever the rendering step would render to screen.

[ilexp]

Note: Consider how the editor / Camera Views would interact with the new setup. Does it make sense to allow users to specify a Rendering and / or Camera Setup to use for every Camera View?

[ilexp]

Note: Consider allowing to configure a rendering step as to whether the RenderTarget to bind should be auto-resized to the TargetResolution before rendering. This could be the default setting, while it would still be possible to assert that the Resource-specified size should be used.

[ilexp]

Progress

• Set up RenderSetup Resource and RenderStep class.
• Added an icon for RenderSetup Resources.

[ilexp]

Progress

• Started updating Camera class and dependencies.

Immediate ToDo

• Introduce temporary / external rendering steps to Camera as outlined above using a before / after ID ordering and hardcoded special IDs for first and last.
• Use them in the editor to specify the additional rendering steps.
• Test camera setup in-game.
• Test camera setup with various editor layers and states.
• Introduce a "base RenderTarget" to cameras which will be used when no rendering target is specified in a pass.
• Introduce overrideable rendering setup (foreach camera) as outline above.
• Introduce auto-resize for RenderTargets used in RenderSetup (optionally keeping aspect ratio?)
• Introduce "blit-fit" mode that specifies how an input texture will be fit to the target size in a RenderStep (center, stretch, fit, fill)
• Introduce a RenderSetup setting on a per-CamView basis.
• Introduce Camera viewports (issue #433) and move that issue to the v3.0 milestone.
• Create a new sample project for different rendering setups and use this to test and polish it all.
  • Post-processing shader.
  • Upscaled pixel art rendering.

[ilexp]

Progress

• "Additional render steps" for Camera instances and used that in the editor instead of the old API. This also fixes the build on the render setup branch.
• Camera base Target that is used whenever a rendering step would otherwise target the screen.
• Default rendering setup specifiable via DualityAppData.
• RenderSetup API to allow overriding custom scene rendering code.

Immediate ToDo

• Introduce auto-resize for RenderTargets used in RenderSetup (optionally keeping aspect ratio?)
• Investigate why there is an InvalidOperation error in OpenGL when switching the antialiasing mode of a currently used RenderTarget in the editor.
• Introduce "blit-fit" mode that specifies how an input texture will be fit to the target size in a RenderStep (center, stretch, fit, fill)
• Introduce a RenderSetup setting on a per-CamView basis.
• Introduce Camera viewports (issue #433) and move that issue to the v3.0 milestone.
• Create a new sample project for different rendering setups and use this to test and polish it all.
  • Post-processing shader.
  • Upscaled pixel art rendering.

[ilexp]

Progress

• Optional auto-resize of a selection of RenderTargets in RenderSetup resources to match output / window size.
• TargetResize math on the master branch to help with the common "resize X to fit Y" problem.
• Fixed the InvalidOperation error in the backend when changing RenderTarget samples while in use.

Immediate ToDo

• Introduce "blit-fit" mode that specifies how an input texture will be fit to the target size in a RenderStep (center, stretch, fit, fill)
• Introduce a RenderSetup setting on a per-CamView basis.
• Introduce Camera viewports (issue #433) and move that issue to the v3.0 milestone.
  • Also, make sure the viewportRect parameter that is passed all over works properly, especially with regard to Y position and height. Would it make sense to replace it with a Point2 outputSize?
• Create a new sample project for different rendering setups and use this to test and polish it all.
  • Post-processing shader.
  • Upscaled pixel art rendering.

[ilexp]

Progress

• Introduced InputResize setting to RenderStep that specifies how a given input will be scaled in order to fit the expected output size.

Immediate ToDo

• Untangle Camera viewports.
  • There is one projection size of rendering, i.e. what is used to set up camera matrices. This could be DualityApp.TargetResolution.
  • The viewportRect parameter passed to rendering methods in Camera and to DrawDevice via property will only adjust the target viewport, stretching the image that is rendered.
  • Make sure the viewportRect parameter works properly, especially with regard to Y position and height.
  • Consider specifying the nominal size for camera matrix setup via parameter as well. Don't propagate a central TargetResolution where it can be avoided.
• Adjust Camera.RenderPickingMap to match its active rendering setup's screen space transformations.
  • Example A: Render to target X, then render that one to screen in stretched mode.
  • Example B: Render to target X, then render that one to target Y (different size) in stretched mode, then render that one to screen in stretched mode again.
  • Will probably have to walk the rendering path backwards from screen to find the final transform.
  • Use that transform to determine picking size and viewport / projection rects when rendering the picking map.
• Introduce a RenderSetup setting on a per-CamView basis.
• Introduce Camera-local viewports (issue #433) and move that issue to the v3.0 milestone.
• Create a new sample project for different rendering setups and use this to test and polish it all.
  • Post-processing shader.
  • Upscaled pixel art rendering.

[ilexp]

Progress

• DrawDevice.ViewportRect is now properly translated to OpenGL coordinates in the backend, so viewports on that level are fully supported now.
• Introduced a DrawDevice.TargetSize property that defined the size of the rendered image, i.e. the one that is used for generating the projection matrices. It is now decoupled from ViewportRect, so adjusting the viewport stretches and scales the rendered content as expected.
• Added graphics backend API for informing it about the size of an external backbuffer that is used instead of a window that it created explicitly.

Immediate ToDo

• Adjust Camera render API to specify viewport and target image size via separate parameters.
• Adjust Camera.RenderPickingMap to match its active rendering setup's final target-to-screen transformations.
  • Assume the biggest target is the main one.
  • Apply the rendering step's target-resize operation and compare it to the available rendering area.
  • Use that transform to determine picking size and viewport / projection rects when rendering the picking map.
• Allow AutoResize targets to have a scale vector, so it will be possible to set up ping-pong downscale rendering chains for bloom effects and similar.
• Introduce a RenderSetup setting on a per-CamView basis.
• Introduce Camera-local viewports (issue #433) and move that issue to the v3.0 milestone.
• Create a new sample project for different rendering setups and use this to test and polish it all.
  • Post-processing shader.
  • Upscaled pixel art rendering.

[ilexp]

Progress

• Adjusted camera rendering API to separate viewport and target image size in parameters.
• Editor picking is now done based on a half-resolution rendering of the scene to improve performance, hopefully without noticeable drawback.
• Investigated automatically adjusting picking maps to match image transformations present in the rendering setup. Turned out, this doesn't really help since editor-overlay drawing is unscaled, and the feature would be hacky at best and only serve a rather obscure use case. Skipping this for now.
• Extended auto-resize feature for render targets that are part of the rendering setup to allow scaling individual targets.
• Introduced an optional "forced rendering size" to DualityAppData settings that can be used to maintain a fixed image size across multiple resolutions by auto-adjusting the viewport rect.

Immediate ToDo

• Introduce a RenderSetup setting on a per-CamView basis.
• Introduce Camera- and RenderStep-local viewports (issue #433) and move that issue to the v3.0 milestone.
• Create a new sample project for different rendering setups and use this to test and polish it all.
  • Post-processing shader.
  • Upscaled pixel art rendering.

[ilexp]

Progress

• It is now possible to switch the used RenderSetup per-CamView.
• Introduced a new per-Camera and per-RenderStep TargetRect property which acts as issue #433 intended.

Immediate ToDo

• Create a new sample project for different rendering setups and use this to test and polish it all.
  • Post-processing shader.
  • Upscaled pixel art rendering.

[ilexp]

Progress

• Added a new sample project for rendering setups.
  • Upscaled fixed-resolution / pixel art rendering.
  • Post-processing shader.
• Various camera tweaks.
• Merged back the feature branch extract-render-setup-wip to develop-3.0.

Done. Closing this.