awslabs / sagemaker-debugger

Amazon SageMaker Debugger provides functionality to save tensors during training of machine learning jobs and analyze those tensors
Apache License 2.0
158 stars 82 forks source link

How do I install protobuf3 compiler and runtime correctly? #171

Open NihalHarish opened 4 years ago

NihalHarish commented 4 years ago

If you see the error messages E ModuleNotFoundError: No module named 'smdebug.core.tfevent.proto.types_pb2'1 or ERROR: Compiling summary protocol buffers failed. You will not be able to use smdebug. Please make sure that you have installed protobuf3 compiler and runtime correctly., it is most likely that you haven't installed the correct version of protobuf.

We provide a simple script that downloads and install the version of protobuf required by sagemaker-debugger.

To install protobuf using this script, simply run.

sh config/protoc_downloader.sh

larroy commented 4 years ago

https://github.com/awslabs/sagemaker-debugger/pull/172

sonalimedani commented 4 years ago

Hi,

I am unable to run that shell script. I am trying a blog https://aws.amazon.com/blogs/machine-learning/pruning-machine-learning-models-with-amazon-sagemaker-debugger-and-amazon-sagemaker-experiments/ and I got this error on line "from smdebug.trials import create_trial":

[image: image.png]

So I tried to install protoc using the shell script in #171 https://github.com/awslabs/sagemaker-debugger/issues/171 but I got this error:

[image: image.png]

I would really appreciate if you can help me. Thanks.

Best, Sonali.

On Fri, May 1, 2020 at 10:36 PM Rahul Huilgol notifications@github.com wrote:

Reopened #171 https://github.com/awslabs/sagemaker-debugger/issues/171.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/awslabs/sagemaker-debugger/issues/171#event-3294918836, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKRB6SUL3LMDFC4SWAGELVLRPOBMBANCNFSM4LB3V7YQ .

Vikas-kum commented 4 years ago

Hi, I am unable to run that shell script. I am trying a blog https://aws.amazon.com/blogs/machine-learning/pruning-machine-learning-models-with-amazon-sagemaker-debugger-and-amazon-sagemaker-experiments/ and I got this error on line "from smdebug.trials import create_trial": [image: image.png] So I tried to install protoc using the shell script in #171 <#171> but I got this error: [image: image.png] I would really appreciate if you can help me. Thanks. Best, Sonali. On Fri, May 1, 2020 at 10:36 PM Rahul Huilgol @.***> wrote: Reopened #171 <#171>. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#171 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKRB6SUL3LMDFC4SWAGELVLRPOBMBANCNFSM4LB3V7YQ .

I can't see the error as images are not attached. Can you please copy paste the error here?

Also please try this script - https://github.com/awslabs/sagemaker-debugger/blob/master/config/protoc_downloader.sh

sonalimedani commented 4 years ago

Hi, when running the script, I got this:

error: cannot delete old /usr/local/bin/protoc Permission denied error: cannot delete old /usr/local/include/google/protobuf/wrappers.proto Permission denied error: cannot delete old /usr/local/include/google/protobuf/field_mask.proto Permission denied error: cannot delete old /usr/local/include/google/protobuf/api.proto Permission denied error: cannot delete old /usr/local/include/google/protobuf/struct.proto Permission denied error: cannot delete old /usr/local/include/google/protobuf/descriptor.proto Permission denied error: cannot delete old /usr/local/include/google/protobuf/timestamp.proto Permission denied error: cannot delete old /usr/local/include/google/protobuf/compiler/plugin.proto Permission denied error: cannot delete old /usr/local/include/google/protobuf/empty.proto Permission denied error: cannot delete old /usr/local/include/google/protobuf/any.proto Permission denied error: cannot delete old /usr/local/include/google/protobuf/source_context.proto Permission denied error: cannot delete old /usr/local/include/google/protobuf/type.proto Permission denied error: cannot delete old /usr/local/include/google/protobuf/duration.proto Permission denied

[image: PNG image] mail.png https://drive.google.com/file/d/11TnnCzqjbRhBsSsoB24gYB0BMNUfx0TI/view?usp=drivesdk

Please let me know if you can see the image

On Sat, May 2, 2020 at 11:57 AM Vikas-kum notifications@github.com wrote:

Hi, I am unable to run that shell script. I am trying a blog https://aws.amazon.com/blogs/machine-learning/pruning-machine-learning-models-with-amazon-sagemaker-debugger-and-amazon-sagemaker-experiments/ and I got this error on line "from smdebug.trials import create_trial": [image: image.png] So I tried to install protoc using the shell script in

171 https://github.com/awslabs/sagemaker-debugger/issues/171 <#171

https://github.com/awslabs/sagemaker-debugger/issues/171> but I got this error: [image: image.png] I would really appreciate if you can help me. Thanks. Best, Sonali. … <#m1185165097498005788> On Fri, May 1, 2020 at 10:36 PM Rahul Huilgol @.***> wrote: Reopened

171 https://github.com/awslabs/sagemaker-debugger/issues/171 <#171

https://github.com/awslabs/sagemaker-debugger/issues/171>. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#171 (comment) https://github.com/awslabs/sagemaker-debugger/issues/171#event-3294918836>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKRB6SUL3LMDFC4SWAGELVLRPOBMBANCNFSM4LB3V7YQ .

I can't see the error as images are not attached. Can you please copy paste the error here?

Also please try this script - https://github.com/awslabs/sagemaker-debugger/blob/master/config/protoc_downloader.sh

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/awslabs/sagemaker-debugger/issues/171#issuecomment-622975082, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKRB6SUUDAAD6ASOVSM6FB3RPQ7IBANCNFSM4LB3V7YQ .

Vikas-kum commented 4 years ago

I can't see the image, It's not public. Have you tried changing Line 16 and 17 to run with sudo . Also try this : https://stackoverflow.com/questions/35896335/how-can-i-uninstall-protobuf-3-0-0

btw what machine(macOSx, linux) & what version you are using?(uname -a) what is output of command "protoc --version" ?

sonalimedani commented 4 years ago

Hi Vikas,

Thank you so much for your help. I uninstalled and used sudo to unzip (which wasn't working before but somehow it worked this time). T

I am now able to move further. Thank you so much :) Have a great day.

Best, Sonali.

On Sat, May 2, 2020 at 12:24 PM Vikas-kum notifications@github.com wrote:

I can't see the image, It's not public. Have you tried changing Line 16 and 17 to run with sudo . Also try this : https://stackoverflow.com/questions/35896335/how-can-i-uninstall-protobuf-3-0-0

btw what machine(macOSx, linux) & what version you are using?(uname -a) what is output of command "protoc --version" ?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/awslabs/sagemaker-debugger/issues/171#issuecomment-622978875, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKRB6SSGCYSWRWCPY65XTITRPRCNRANCNFSM4LB3V7YQ .