Tasks and more II

[Capital-Asterisk]

this took a while. If I remember correctly, trying to smash together some of the new universe stuff caused the duct tape to fall off of the tagged task system, so a new one was needed.

New task system:

• Uses 'Pipelines' and 'Stages' to more cleanly solves the problem of needing a loop consisting of Task A that needs to run before Task B which can run in parallel with Task C to fill a container and can run in parallel with unrelated Tasks D, E, F, and G but also Task H needs every other task to be complete before moving to Task I while all making sure that tasks don't accidentally run multiple times when they don't need to and still be able to multi-thread the heck out of everything some time in the future. See unit test
• Supports certain nested loop schemes that can be used to more cleanly implement wire/links
• Separate 'Executor' interface to eventually support other ways to run the Task/Pipeline/Stage graph (planned multi-threaded executor)
• Superior in pretty much every aspect compared to the previous 'tag' thing
• Might be codenamed "Sovapine II" (unnecessary information)

Example:

before:

shapeSpawn.task() = rBuilder.task().assign({tgSceneEvt, tgSpawnReq, tgSpawnEntReq, tgTransformNew, tgHierNew}).data(
        "Add hierarchy and transform to spawned shapes",
        TopDataIds_t{           idBasic,             idActiveIds,              idSpawner,             idSpawnerEnts },
        wrap_args([] (ACtxBasic& rBasic, ActiveReg_t& rActiveIds, SpawnerVec_t& rSpawner, EntVector_t& rSpawnerEnts) noexcept
{
    // ...
}));

translation: Task triggered when scene event is called tgSceneEvt. The tgSpawnReq tag means that there are other tasks with tags like tgSpawnMod that must be done before running this task. Similar thing with the other tags.

The wrap_args function adapts the lambda's function signature to accept an array of 'any' containers as arguments, since all application data (named TopData) is stored as one big vector of 'any's. idBasic, idActiveIds, ... are indices to the vector of 'any's.

Tags have dependencies on each other; assigning these can get confusing, especially if loops are involved. Dependencies on loops just don't work. There's also the problem of 'optional branches' or conditions of when tasks are able to run that makes things more confusing. There's also quite a bit of ugly linear-search stuff in there that are not particularly fast.

after:

rBuilder.task()
    .name       ("Add hierarchy and transform to spawned shapes")
    .run_on     ({tgShSp.spawnRequest(UseOrRun)})
    .sync_with  ({tgShSp.spawnedEnts(UseOrRun), tgCS.hierarchy(New), tgCS.transform(New)})
    .push_to    (out.m_tasks)
    .args       ({      idBasic,                  idSpawner })
    .func([] (ACtxBasic& rBasic, ACtxShapeSpawner& rSpawner) noexcept
{
    // ...
});

translation: This task runs when the spawnRequest pipeline is on the UseOrRun stage. Task must run once spawnedEnts is on its UseOrRun stage, hierarchy is on its New stage, and transform is on its New stage.

Each 'Pipeline' is a sequence of 'Stages' that are each run one-by-one. Tasks are set to run when a pipeline reaches a certain stage, and can be 'synchronized' to block and only run when other pipelines reach a certain stage. Pipelines can only advance to the next stage when all the tasks running on them are done, and all tasks synchronized with them are done too.

Pipelines can be parented to each other, forming a tree of pipelines. This allows supporting things like having a loop, where pipelines within the loop can sync with each other, and also properly handle cases of these pipelines depending on things outside of the 'loop scope' and vise versa. This gets confusing but the new system is capable of handling it.

Pipeline stages are defined as enums:

/**
 * @brief Continuous Containers, data that persists and is modified over time
 */
enum class EStgCont : uint8_t
{
    Prev,
    ///< Previous state of container

    Delete,
    ///< Remove elements from a container or mark them for deletion. This often involves reading
    ///< a set of elements to delete. This is run first since it leaves empty spaces for new
    ///< elements to fill directly after

    New,
    ///< Add new elements. Potentially resize the container to fit more elements

    Modify,
    ///< Modify existing elements

    Ready
    ///< Container is ready to use
};

The pipeline stuff is similar to having mutexes and threads. One problem with mutexes, is that there's no way to know if a function is going to get blocked by a locked mutex without running the function, leading to a large number of blocked threads. Storing the 'blocking information' externally as pipelines/stages/tasks means that some executor code can very effectively coordinate which tasks can run in parallel using a fixed number of threads. It's also possible to analyze critical paths within the graph of pipelines/stages/tasks and assign priorities to tasks.

Other stuff

• Broke every test scene except for the physics test scene
• Renamed ActiveApplication to MagnumApplication because wtf is an active?
• Full renderer/scene separation for real this time
• Add "DrawEnt" to separate ActiveEnt from rendering. This makes it easier to draw things without needing ActiveEnt.
• Add "Thrust Indicator" system for visualizing rocket thrust (currently commented out)

again, it's probably not worth trying to review the whole thing (or any at all). since...

• quite a bit more code is going to be microwaved in the next few days
• my code is already rather high quality (depending on who you ask of course, but it's definitely better than my previous code from a few months ago)
• I'd like to get this project off the ground a little quicker and have something that actually looks cool