awslabs / sagemaker-debugger

Amazon SageMaker Debugger provides functionality to save tensors during training of machine learning jobs and analyze those tensors
Apache License 2.0
161 stars 83 forks source link

Pin pyinstrument version to 3.4.2 #514

Closed ndodda-amazon closed 3 years ago

ndodda-amazon commented 3 years ago

Description of changes:

Pyinstrument 4.0.0 introduces breaking changes that cause Python profiling to fail with this error:

RuntimeError: Could not find app.js. If you are running pyinstrument from a git checkout, run 'python setup.py build' to compile the Javascript (requires nodejs).

It is unclear what the root cause is, because smdebug consumes pyinstrument consumed directly from PyPi and not from a git checkout. I was able to reproduce this error without smdebug on the TF 2.4 DLC images. See the issue I created against pyinstrument here: https://github.com/joerick/pyinstrument/issues/137.

Until this issue is debugged, the version of pyinstrument will be pinned to 3.4.2.

Style and formatting:

I have run pre-commit install && pre-commit run --all-files to ensure that auto-formatting happens with every commit.

Issue number, if available

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

codecov-commenter commented 3 years ago

Codecov Report

Merging #514 (5015ff5) into master (b9765cb) will decrease coverage by 1.79%. The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #514      +/-   ##
==========================================
- Coverage   75.15%   73.36%   -1.80%     
==========================================
  Files         127      117      -10     
  Lines       11110    10584     -526     
==========================================
- Hits         8350     7765     -585     
- Misses       2760     2819      +59     
Impacted Files Coverage Δ
smdebug/pytorch/utils.py 45.23% <0.00%> (-35.72%) :arrow_down:
smdebug/mxnet/utils.py 18.75% <0.00%> (-31.25%) :arrow_down:
smdebug/rules/action/message_action.py 81.92% <0.00%> (-15.67%) :arrow_down:
smdebug/xgboost/utils.py 0.00% <0.00%> (-14.76%) :arrow_down:
smdebug/profiler/tf_profiler_parser.py 54.54% <0.00%> (-11.58%) :arrow_down:
smdebug/rules/action/stop_training_action.py 54.68% <0.00%> (-7.82%) :arrow_down:
smdebug/core/reduction_config.py 88.31% <0.00%> (-7.80%) :arrow_down:
smdebug/mxnet/hook.py 83.33% <0.00%> (-6.82%) :arrow_down:
smdebug/mxnet/collection.py 73.33% <0.00%> (-6.67%) :arrow_down:
smdebug/core/singleton_utils.py 79.41% <0.00%> (-5.89%) :arrow_down:
... and 25 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update b9765cb...5015ff5. Read the comment docs.