[critical path] Add save and restore for cp_graph

facebookresearch / HolisticTraceAnalysis

A library to analyze PyTorch traces.

http://hta.readthedocs.io

MIT License

293 stars 40 forks source link

[critical path] Add save and restore for cp_graph #126

Closed briancoutinho closed 5 months ago

briancoutinho commented 6 months ago

What does this PR do?

Enable save and restore of critical path

Save

zip_file = cp_graph.save(out_dir="/tmp/my_saved_cp_graph")

Restore

from hta.analyzers.critical_path_analysis import restore
rest_graph = restore_cpgraph(zip_filename=zip_file, t_full=analyzer.t, rank=rank)

Screenshot 2024-05-01 at 6 12 10 PM

Updated documentation in example notebook

https://github.com/facebookresearch/HolisticTraceAnalysis/blob/d36b08de7d91d4435457b376e53fc620c1b2ef2d/examples/experimental/critical_path_analysis.ipynb

Unit Test

pytest tests/test_critical_path_analysis.py -k test_critical_path_breakdown_and_save_restore

Before submitting

[ ] Was this discussed/approved via a Github issue? (no need for typos, doc improvements)
- [x] N/A
[x] Did you write any new necessary tests?
- [ ] N/A
[] Did you make sure to update the docs?
- [x] N/A
[x] Did you update the changelog?
- [] N/A

facebook-github-bot commented 6 months ago

@briancoutinho has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

codecov-commenter commented 6 months ago

Codecov Report

Attention: Patch coverage is 97.77778% with 2 lines in your changes are missing coverage. Please review.

Project coverage is 91.01%. Comparing base (4ad2ea6) to head (d2d1348). Report is 2 commits behind head on main.

Files	Patch %	Lines
hta/common/execution_trace.py	50.00%	2 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #126 +/- ## ========================================== + Coverage 90.84% 91.01% +0.16% ========================================== Files 52 52 Lines 4968 5084 +116 ========================================== + Hits 4513 4627 +114 - Misses 455 457 +2 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

facebook-github-bot commented 6 months ago

@briancoutinho has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.