aws / amazon-sagemaker-examples

Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.
https://sagemaker-examples.readthedocs.io
Apache License 2.0
10.09k stars 6.76k forks source link

[Bug Report] cp: cannot stat '/root/.keras/keras_mxnet.json': No such file or directory #3317

Open JoanWu5 opened 2 years ago

JoanWu5 commented 2 years ago

Link to the notebook https://github.com/aws-samples/reinvent2018-srv404-lambda-sagemaker/blob/master/training/sms_spam_classifier_mxnet.ipynb

Describe the bug I am writing a lifecycle configuration script, I can run the script well on a notebook terminal: image However, when I try to run it on lifecycle, it failed with the linesource "/home/ec2-user/anaconda3/bin/activate" mxnet_p36, the log reports: image

To reproduce The lifecycle script is:

set -e
NOTEBOOK_FILE="/home/ec2-user/SageMaker/smlambdaworkshop/training/sms_spam_classifier_mxnet.ipynb"
echo "activate mxnet_p36"
source "/home/ec2-user/anaconda3/bin/activate" mxnet_p36
echo "execute jupyter notebook"
jupyter trust "$NOTEBOOK_FILE"
nohup jupyter nbconvert --to notebook --inplace --execute "$NOTEBOOK_FILE" --ExecutePreprocessor.kernel_name=python3 --ExecutePreprocessor.timeout=-1&
echo "finish executing jupyter notebook"

Logs image

howieraem commented 2 years ago

Hi, have you resolved the problem or found any workaround?

JoanWu5 commented 2 years ago

The workaround is the using the following script: echo '{ "floatx": "float32", "epsilon": 1e-07, "backend": "mxnet", "image_data_format": "channels_last" }' | sudo tee -a /root/.keras/keras_mxnet.json

source /home/ec2-user/anaconda3/bin/activate mxnet_p36 && pip install mxnet

But I hope the developer team can solve this issue since keras_mxnet.json isn't necessary in this situation