ComputationalRadiationPhysics / redGrapes

Resource-based, Declarative task-Graphs for Parallel, Event-driven Scheduling :grapes:
https://redgrapes.rtfd.io
Mozilla Public License 2.0
20 stars 5 forks source link

Task Template refactor #58

Closed ikbuibui closed 6 months ago

ikbuibui commented 7 months ago

Refactor RedGrapes away from macro based configuration, to using a templated task type Change the init method Move away from a lot of global variables Fix the MPI example. MPI calls are now executed from the same thread We can now have multiple thread pools/single threads with different kinds of workers Bump to C++ 20 and start using concepts A lot more

ikbuibui commented 7 months ago
1. Using [&] in all lambdas should be considered bad practice when writing redGrapes applications since this makes is _very easy_ to introduce race conditions or lifetime issues. It is much safer to write out all capture-by-references explicitly.  Requiring to capture the redGrapes-context in each task therefore degrades usability.

Yes, the capture by reference of the "redGrapes-context" should be done explicitly. However it currently isn't strictly required to capture it in each task, only in task which need to access methods in the redGrapes class inside the task from the user side, like emplace (for child tasks), scope_depth, create_event etc.

2. It increases the size of each task by another 8-bytes to store the reference. Having a singleton is more efficient from this perspective, as we have seen that the memory-footprint of tasks is crucial for cache-utilization and thus overall performance.

I haven't profiled the code yet, so I appreciate this insight.

Further, I saw that we now add another reference to the scheduler-object in each task , bloating the task-struct even more. But thats fine for now, since we could probably optimize this later by using an 8-bit Index instead and storing the references to the scheduler objects in a global map.

That is a good point and I also dislike storing the reference to the scheduler in the task. I was thinking of storing the scheduler information only as a type with the task. But this leads to some other complications so I havent decided on a way forward yet. Thanks for the suggesion

At least on my system, the examples 1_resources , 4_refinements, game_of_life and mpi all fail to compile with the following error:

redGrapes/redGrapes/resource/access/area.hpp:75:25: error: call of overloaded 'format_to(fmt::v10::basic_format_context<fmt::v10::appender, char>::iterator, const char [46], const std::array<long unsigned int, 2>::value_type&, const std::array<long unsigned int, 2>::value_type&)' is ambiguous
   75 |         return format_to(ctx.out(), "{{ \"area\" : {{ \"begin\" : {}, \"end\" : {} }} }}", acc[0], acc[1]);

Thanks for the report. I was able to reproduce it with clang. Will fix it.

ikbuibui commented 7 months ago

Implemented changes and bumped minor version of RedGrapes

michaelsippel commented 6 months ago

Is @psychocoderHPC fine with the updated changes?

ikbuibui commented 6 months ago

Is @psychocoderHPC fine with the updated changes?

Broadly yes, he will review the updates next week before merging.