search

Point72 / csp

csp is a high performance reactive stream processing library, written in C++ and Python
https://github.com/Point72/csp/wiki
Apache License 2.0
150 stars 27 forks source link

Runtime profiling of the Graph #279

Closed Roh-codeur closed 3 weeks ago

Roh-codeur commented 3 weeks ago

Hi

thanks for sharing this, this is a brilliant library.

Is your feature request related to a problem? Please describe. I am looking at the cap wiki page for profiling. I couldn't find something in the library which would offer stats while the graph is running in realtime(or simulation). This is extremely useful because it would help us identify hotspots at runtime. A brilliant library which does it is TimerOutputs (has an extremely small overhead) https://github.com/Point72/csp/wiki/csp.profiler-API https://github.com/KristofferC/TimerOutputs.jl

Describe the solution you'd like An optimal solution for this would belong in C++ codebase. Unfortunately beyond my expertise

Describe alternatives you've considered I am trying to put this together in Python. This is the first node. I use the state to keep a track of the last cycle time to current cycle time and calculate stats.

Additional context Something like the below, from C++ code would be ideal. https://github.com/KristofferC/TimerOutputs.jl

thanks a lot for considering this

ta!

AdamGlustein commented 3 weeks ago

Hey! Yes, we have built-in real-time profiling during graph execution, and all the timing is done in C++. You can setup a local HTTP port and write the results there, here is the docs on it: https://github.com/Point72/csp/wiki/Profile-CSP-Code#profiling-a-real-time-cspgraph.

I do see that we are missing this in the API docs though, so thanks for bringing it to our attention!

Roh-codeur commented 3 weeks ago

thanks Adam!