Graphics abstraction: blocks

[hadronized]

• [x] GPU inputs.
• [x] GPU outputs.
• [ ] GPU blocks.
  • [ ] A block should be able to update when its GLSL code updates.
  • [ ] Blocks should be composable; e.g. if I have two blocks P: A -> B and Q: B -> C, I want combinators like P ~ Q: A -> C.
  • [x] Blocks should be instantiable in different ways. A block is just a generic computation occurring in the GPU via shader code. We can instantiate them to work on vertices, fragments or more complex objects.

[hadronized]

On GPU blocks

Fundamentals

Blocks are composed of three main parts:

• Inputs, named types without external semantics.
• Outputs, named types without external semantics.
• GLSL code, representing a function that takes the block’s inputs as arguments and returns the block’s outputs as outputs.

Combinators

There are some block transformations that are combinators:

• Composition: composing a block X: A -> B with Y: B -> C yields a block Z: A -> C. Typically, that kind of yielded block will simply get its inputs from X, its outputs from Y and internal representation will be injected in the code. It’s important to note that the intermediate C (both inputs and outputs) must remain used as-is, so that sharing is possible – imagine that several blocks use the same block.
• Mapping: mapping a function B -> C on a block X: A -> B yields a block Y: A -> C.
• Contravariant mapping: contra mapping a function A -> C on a block X: A -> B yields a block Y: C -> B.
• Pro mapping: contra mapping a function A -> C and mapping a function B -> D on a a block X: A -> B yields a block Y: C -> D.
• Blending: blend one or several blocks into a single one; i.e. X: A -> B, Y: C -> D, Z: E -> F, blend(X, Y, Z) = δ: (A, C, E) -> G.

What’s in the box?

The GLSL is just a simple AST. That is, a Vec<ExternalDeclaration> – see my glsl crate for further details. The idea is that it must define:

• Its Inputs.
• Its Outputs.
• The Vec<ExternalDeclaration> that makes its code:
  • All of the ExternalDeclarations are brought into scope.
  • But the entry point is always the call function.
  • That function takes as input the struct representation of the block’s inputs and outputs a struct representation of the block’s outputs.
  • In combinators cases, hidden structs are injected as ExternalDeclaration and brought into scope automatically. For instance, given X: A -> B, Y: B -> C, the blockX ~ Ywill have as inputsAand outputsCand its internal code will have the struct definition ofBas well as the twocallfunctions fromXandY`.

Unresolved: identifiers?

In order to authorize a correct naming of structs and functions, they should be named randomly. For instance, the call functions must carry a prefix or suffix. We can use the name of the box for that, but that’s not enough. I think that we need somekind of a context here in order to be able to get a new render block identifier each time a new one must be created. This will prevent nasty conflicts.

[hadronized]

Do we want to be able to reference the call functions from dependent blocks? If so, how so? Also, how do we reference other input / output types from the current block?

X: A -> B
Y: B -> C
Z: A -> C

Here, the first B (output) is not the exact same as the second B (input) if you strictly generate the GLSL code for X, Y and Z. We need to find a way to share B between X and Y for this to work.

[hadronized]

Maybe a lead for implementing composition:

1. Set the inputs of the composition block to A.
2. Set the outputs of the composition block to C.
3. Generate the GLSL code:
   1. Change every occurrence to the Out type in the code of X to InOut_{name_of_composition_block}.
   2. Do the same for In from Y.
   3. Now, inject a single copy of the type (InOut_{name_of_composition_block}) as ExternalDeclaration, then both mangled code from X and Y. We get two functions call_x and call_y that now share the same input / output type.
   4. Implement call, which is now trivial.
4. Profit.