mit-pdos / noria

Fast web applications through dynamic, partially-stateful dataflow
Apache License 2.0
4.99k stars 244 forks source link

Dynamic load profiling #90

Open jonhoo opened 6 years ago

jonhoo commented 6 years ago

It'd be really cool if the system kept stats about how "busy" different parts of the graphs were. For example, how many records they're processing per second or how long they spend idle between every time they're woken up. This might help to identify performance regressions and opportunities for optimization, but could also have a wider impact by allowing the controller to re-shard or replicate nodes in response to increase/decreased load.