search

charmplusplus / charm

The Charm++ parallel programming system. Visit https://charmplusplus.org/ for more information.
Apache License 2.0
207 stars 50 forks source link

Unify Data Collection in Charm++ #1059

Open rbuch opened 8 years ago

rbuch commented 8 years ago

Original issue: https://charm.cs.illinois.edu/redmine/issues/1059

Currently, things like load balancing, PICS, BigSim, and tracing (for Projections) all collect data in their own unique way. This causes unnecessary duplication of code and data. This will likely get worse in the future as we add more detail for things like heterogeneous load balancing and energy aware scheduling.

Thus, we should move towards a model of unified data collection. One module in Charm++ should collect information in a sort of generic database, and those services that use the data can get it from this unified system. This should help reduce the complexity of the code, memory usage, and overhead and simplify the development of new runtime features.

A wrinkle in this is that it will be a lot of software engineering work for no externally visible impact, but it's work worth doing.

PhilMiller commented 5 years ago

Original date: 2017-10-11 20:19:16

Ronak, could you make the scheduling decision on this, and maybe identify incremental subtasks that could be scheduled sooner if the overall thing won't be done so soon?