awslabs / sagemaker-debugger

Amazon SageMaker Debugger provides functionality to save tensors during training of machine learning jobs and analyze those tensors
Apache License 2.0
161 stars 83 forks source link

Disable gradient tape profiling tests #525

Closed adimux closed 2 years ago

adimux commented 2 years ago

Description of changes:

Tests related to profiling using gradient tape are failing and blocking the TF2.7 release. Since this feature has never been released, we can safely disable the tests, until we make a decision about supporting this feature.

Style and formatting:

I have run pre-commit install && pre-commit run --all-files to ensure that auto-formatting happens with every commit.

Issue number, if available

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

codecov-commenter commented 2 years ago

Codecov Report

Merging #525 (423c4c2) into master (b4dd4c1) will decrease coverage by 6.42%. The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #525      +/-   ##
==========================================
- Coverage   77.60%   71.18%   -6.43%     
==========================================
  Files         127      117      -10     
  Lines       11111    10585     -526     
==========================================
- Hits         8623     7535    -1088     
- Misses       2488     3050     +562     
Impacted Files Coverage Δ
...debug/profiler/analysis/notebook_utils/__init__.py 0.00% <0.00%> (-100.00%) :arrow_down:
smdebug/mxnet/utils.py 18.75% <0.00%> (-59.38%) :arrow_down:
...g/profiler/analysis/notebook_utils/training_job.py 0.00% <0.00%> (-50.00%) :arrow_down:
smdebug/xgboost/utils.py 0.00% <0.00%> (-41.54%) :arrow_down:
smdebug/xgboost/__init__.py 0.00% <0.00%> (-40.00%) :arrow_down:
smdebug/mxnet/graph.py 22.58% <0.00%> (-37.10%) :arrow_down:
smdebug/pytorch/utils.py 45.23% <0.00%> (-35.72%) :arrow_down:
smdebug/trials/profiler_trial.py 24.61% <0.00%> (-32.31%) :arrow_down:
smdebug/tensorflow/keras.py 57.42% <0.00%> (-30.26%) :arrow_down:
smdebug/rules/action/stop_training_action.py 54.68% <0.00%> (-26.57%) :arrow_down:
... and 56 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update b4dd4c1...423c4c2. Read the comment docs.

adimux commented 2 years ago

xgboost tests are failing in the nightly build of sagemaker-debugger, so this PR is ok to merge

Roshrini commented 2 years ago

LGTM. Thanks