Pprof module extension

[JackyWYX]

Description

Extend the harmony node's Pprof service with some advanced developer options.

Context

Pprof is a golang tool for resource profiling, e.g. CPU, memory, goroutine, e.t.c, which is powerful for developer to discover and resolve performance and memory leak issues.

Currently, Harmony program only support profiling over HTTP connection. This results in two limitations:

1. If a developer want to do pprof on a certain machine, an additional port (default as 6060) is needed to be opened. This will add potential risk to the network security.
2. The profiling over HTTP is runtime, which means some critical data of some historic period can be missing, making it hard to debug and reproduce the problematic scenario.

Thus, an extended version of pprof service is needed.

Requested features

The extended version pprof service should have the following features:

1. Other than profiling over HTTP, periodically dump all kinds of pprof data to local disk according to user settings.
2. Add flags for pprof parameters to cmd as necessary.
3. Add fields of pprof parameters to harmony.conf (code) which is the config file loaded when starting node.
4. Add harmony.conf migration logic for pprof new fields to enable backward compatibility for loading config file of old version. (Reference)

Suggested approach

It would be suggested to build pprof into a self-organized module, and add it to service.Manager. If you have any better approach, any discussions are welcomed.

Acceptance Criteria

• [ ] Build a pprof module, registered into service.Manager.
• [ ] Pprof can provide profiling over HTTP (as the current implementation).
• [ ] Pprof can dump data files to local disk periodically as user settings.
• [ ] Add cmd flags as necessary. Some very detailed flags should be hidden from --help message for non-developer users.
• [ ] Update harmony.conf data structure and add config file migration logic for old versioned config file.
• [ ] All changes are backward compatible.
• [ ] Fix existing unit test cases as necessary. Unit test coverage >=80% for new code.
• [ ] A complete manual / cheatsheet of how to do analysis / visualization through both HTTP and dumped pprof files.
• [ ] End -to end test with report. (An example)
  • Running node with legacy config file.
  • Running node with latest config file.
  • Example pprof output data.

Reward

USD $3,000 equivalent of Harmony ONE token.

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

This issue now has a funding of 29298.8405 ONE (3008.9 USD @ $0.1/ONE) attached to it.

• If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
  • Want to chip in? Add your own contribution here.
  • Questions? Checkout Gitcoin Help or the Gitcoin's Discord
  • $1,713,312.66 more funded OSS Work available on the Gitcoin Issue Explorer

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work has been started.

These users each claimed they can complete the work by 1 month, 3 weeks from now. Please review their action plans below:

1) yashagarwal9 has started work.

Will Update

Learn more on the Gitcoin Issue Details page.

[19Clemente92]

nice