Better Profiler

[chenhao94]

There are 2 possible works we can do to improve our profiler:

• Performance trend over time. We can dump the statistical details somewhere periodically (by recorder with Protobuf/JSON encoded stat?) somewhere, and make a (Python?) tool to aggregate the result or show the trends.
• Async events profiling. When some input comes, it may trigger a chain of events to process the input, while the events are asynchonous to each other. We may care about the latency of processing the input while understanding how long does each event take. So we may write information of each event, including time, input ID, etc., to some journal, so that other tools (python?) may analyze the profiling results later. This can also be done by our recorder w/ json/protobuf encoded event info. Or if there are some existing 3rd-party tools that will also be great.

While the analyzing tools may take time to implement, we can start from dumping those journals somewhere.

[chenhao94]

@xkszltl ideas or comments?

[xkszltl]

LGTM