awslabs / sagemaker-debugger

Amazon SageMaker Debugger provides functionality to save tensors during training of machine learning jobs and analyze those tensors
Apache License 2.0
161 stars 83 forks source link

TF 2.2 documentation update #322

Closed mchoi8739 closed 4 years ago

mchoi8739 commented 4 years ago

Description of changes:

Update TF documentation, pointing out the newest feature with that latest Debugger pySDK and smdebug release.

Style and formatting:

I have run pre-commit install to ensure that auto-formatting happens with every commit.

Issue number, if available

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

codecov-commenter commented 4 years ago

Codecov Report

Merging #322 into master will decrease coverage by 21.14%. The diff coverage is n/a.

Impacted file tree graph

@@             Coverage Diff             @@
##           master     #322       +/-   ##
===========================================
- Coverage   85.29%   64.15%   -21.15%     
===========================================
  Files          86       86               
  Lines        6352     6352               
===========================================
- Hits         5418     4075     -1343     
- Misses        934     2277     +1343     
Impacted Files Coverage Δ
smdebug/tensorflow/__init__.py 0.00% <0.00%> (-100.00%) :arrow_down:
smdebug/tensorflow/constants.py 0.00% <0.00%> (-100.00%) :arrow_down:
smdebug/tensorflow/singleton_utils.py 0.00% <0.00%> (-100.00%) :arrow_down:
smdebug/tensorflow/collection.py 0.00% <0.00%> (-95.88%) :arrow_down:
smdebug/tensorflow/keras.py 0.00% <0.00%> (-93.59%) :arrow_down:
smdebug/tensorflow/session.py 0.00% <0.00%> (-91.87%) :arrow_down:
smdebug/tensorflow/utils.py 0.00% <0.00%> (-87.77%) :arrow_down:
smdebug/tensorflow/callable_cache.py 0.00% <0.00%> (-82.61%) :arrow_down:
smdebug/tensorflow/base_hook.py 0.00% <0.00%> (-79.01%) :arrow_down:
smdebug/tensorflow/tensor_ref.py 11.29% <0.00%> (-77.42%) :arrow_down:
... and 24 more

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update e38f381...1eee9c6. Read the comment docs.

Vikas-kum commented 4 years ago

On main doc page there is a Note: Limited support of the zero script change experience for TensorFlow 2.2. The tensor collections layers, inputs, outputs, and gradients are currently not available

I think you would need to remove that.

mchoi8739 commented 4 years ago

On main doc page there is a Note: Limited support of the zero script change experience for TensorFlow 2.2. The tensor collections layers, inputs, outputs, and gradients are currently not available

I think you would need to remove that.

I will do after I verify by running the test notebook that I found the image issue with cuda version. I want to make sure if everything working is true for customers.

NihalHarish commented 4 years ago

Can we also add a section on the version and behavior of smdebug shipped on different DLCs

mchoi8739 commented 4 years ago

Confirm that the bug has fixed. Removing the limitation note.

mchoi8739 commented 4 years ago

Can we also add a section on the version and behavior of smdebug shipped on different DLCs

I am not sure what are the different behaviors would be? I thought it works in the same way for all Debugger-supported DLCs. Can you tell me what are those behaviors?

Vikas-kum commented 4 years ago

On main doc page there is a Note: Limited support of the zero script change experience for TensorFlow 2.2. The tensor collections layers, inputs, outputs, and gradients are currently not available I think you would need to remove that.

I will do after I verify by running the test notebook that I found the image issue with cuda version. I want to make sure if everything working is true for customers.

"want to make sure if everything working is true for customers." -> This should be done before the docs merged in and made public. I hope it is working now.

mchoi8739 commented 4 years ago

On main doc page there is a Note: Limited support of the zero script change experience for TensorFlow 2.2. The tensor collections layers, inputs, outputs, and gradients are currently not available I think you would need to remove that.

I will do after I verify by running the test notebook that I found the image issue with cuda version. I want to make sure if everything working is true for customers.

"want to make sure if everything working is true for customers." -> This should be done before the docs merged in and made public. I hope it is working now.

yes, I confirmed that it works now. I tested with an example notebook.

NihalHarish commented 4 years ago

Can we also add a section on the version and behavior of smdebug shipped on different DLCs

I am not sure what are the different behavior would be. I thought it works in the same way for all Debugger-supported DLCs. Can you tell me what are those behaviors?

In the case of TF 2.x containers, inputs, outputs, gradients, and intermediate layer are only saved for the TF 2.2.0 container

mchoi8739 commented 4 years ago

Can we also add a section on the version and behavior of smdebug shipped on different DLCs

I am not sure what are the different behavior would be. I thought it works in the same way for all Debugger-supported DLCs. Can you tell me what are those behaviors?

In the case of TF 2.x containers, inputs, outputs, gradients, and intermediate layer are only saved for the TF 2.2.0 container

ok, updating the note that I put here: https://github.com/mchoi8739/sagemaker-debugger/blob/master/docs/tensorflow.md#debugger-built-in-tensor-collections-for-tensorflow