Open SimonDanisch opened 3 years ago
Hi there! The author of entt
also has a bunch of reading material regarding ECS
it goes from high level design pattern ideas to the deep underlying data structures.
For a first dive into ECS in general I would start with this first of multiple blog posts on the topic: https://skypjack.github.io/2019-02-14-ecs-baf-part-1/
entt
also has a very rich documentation on how to employ it (or ECS more generally): https://entt.docsforge.com/master/entity-component-system/
ECS is fascinating and very powerful. It's very much worth investigating and considering. But it might not be the best solution to any given problem.
Since Simon implemented scatter here, let's take that as an example. Here we currently have:
mutable struct Scatter{N}
positions::Vector{Point{N, Float32}}
color::TorVector{RGBAf0}
marker::TorVector{<: Union{Symbol, Char, Type{Circle}}}
markersize::TorVector{Union{Float32, Vec{N, Float32}}}
markeroffset::TorVector{Vec{N, Float32}}
strokecolor::TorVector{RGBAf0}
strokewidth::TorVector{Float32}
markerspace::Space
transform_marker::Bool
on_update::Observable{Dict{Symbol, Any}}
on_event::Observable{Tuple{Symbol, Any}}
camera::Camera
transformation::Transformation
end
I.e. we have one component that is per scatter element (positions) and a bunch that are TOrVector
, i.e either a single instance "inherited" to each element or a collection with one value per element. The shaders in GLMakie are usually implemented to handle either with structures like {{color_type}} color;
which get resolved when a plot is transformed to a render object.
In my first attempt at translating scatter I made each element an entity. That should allow some cool things, like adjust scatter element 1317 to be transformed to a scattered polygon, element 503 to change color etc without having to recreate the whole scatter. The question then is how to handle components in a way that doesn't duplicate a ton of data and that arrives at a reasonably optimal shader. Consider
# This should have all components only once and end up with a shader that uses no textures/buffers
scatter(lots_of_points, marker = Pixel())
# It would be cool if this ends up with 3 calls of the above
scatter(lots_of_points, color = rand(1:3))
# This needs to switch to a shader that has colors in a texture/buffer (or maybe even vertexarray)
# but should keep other attributes as singular values
scatter(points, color = rand(RGBA, length(points))
# it would be cool if this worked
# this should replace a ~ScatterMarker component with a ~MeshElement/Poly component
# that component should be combined with other such components to draw one merged mesh
entity_pool = scatter(points)
entity_pool.marker[17] = some_polygon
@ffreyer please allow me to critique your proposal from an ECS standpoint. It might be that for the purpose of a plotting library it's preferable to an ECS design.
What you currently propose is a "root object" which is a mutable structure of arrays. This is already very good for performance. But you are restricting what fields a scatter
object may have, thus is only expandable if you add new fields to the scatter
struct.
In an ecs approach (I will use entt
terminology because that's what I'm familiar with) you wouldn't necessarily have a data structure called scatter
, you probably would have a function though. In entt
you have a "root" data structure called the registry
, this registry is essentially an in-memory database, and all your state is supposed to live inside the registry
. You can use this registry
to create entities, which conceptually is nothing more than an index. To each index, you may or may not (also often in batches) assign components, such as Position
, Color
, Marker
, etc... So you can have a scatter
function that creates a batch of entities and assigns them the components which are necessary to draw a scatter plot. The interesting thing then is, when you call the scatter
function on another set of data, you add these to the same registry. You may then also on top call an arrows
function, which as you know consists of a lines
and a scatter
call; and all the respective entities and components are added to the registry.
Notice that the OpenGL call required to draw the lines is different than the one required to draw the arrow heads, but this one in turn is the same which was needed to draw the first two scatter
calls. Now the beauty of ECS, is that now when want to draw your complicated scene, you can easily build your OpenGL draw calls based on the nature of the data required for each one, i.e. all the three scatter
calls can be joined in a single draw call, because you can filter the entities by the components they have. You can ask the registry
to give you all the positions and all the colours of only the scatter
calls. And then you can do a separate lines
OpenGL call for the arrows
tails. And the neatest thing is that all Positions
of all three scatter
calls will live contiguously in memory, allowing for very cheap copying to the GPU.
Where ECS shines is obviously in a game engine, because there you want to simulate a bunch of different things but with many overlapping aspects. Because you can associate different behaviours to the existence or absence of certain components on an entity.
So maybe to summarise the approach, different high level API/recipe calls would create a bunch of entities and give them their meaningful attributes. And then the draw calls would looks for the common relevant attributes, agnostic from which recipe they came from.
I hope this is clear enough.
The struct is what Simon has here, it was just supposed to be a reference for what components we need.
My current idea of an ECS (which may very wall be bad) is that a scatter plot splits into one entity per element, each of which can have a position
, color
, offset
, transform
, ... component.
To each index, you may or may not (also often in batches) assign components
I guess something like scatter should be batched from the start? I.e. we should have a Color
and a BatchedColor
component depending on whether one calls scatter(points, color = RGBA(...))
or scatter(points, color = rand(RGBA, length(points)))
and only one entity per plot call?
A few people got interested in using an ECS for Makie (E.g. @ffreyer, @louisponet).
I tried to extract some useful comments about pros and cons from the slack thread:
@c42f:
@sunbubble
@c42f
@sunbubble
@louisponet created https://github.com/louisponet/Overseer.jl, which seems like a good starting point for an ECS in Julia.
Examples: https://github.com/louisponet/Glimpse.jl/blob/master/examples/boids.jl https://github.com/c42f/Gameoji#readme