Lifecycle config cannot run notebook that imports sagemaker, boto3 libraries

[Dan-Treacher]

I have the following lifecycle config:

#!/bin/bash
sudo -u ec2-user -i <<'EOF'

source activate python3
pip install requests==2.21.0
echo "Installing boto3"
pip install boto3
echo "Starting nohup command"
nohup jupyter nbconvert /home/ec2-user/SageMaker/TestNotebook.ipynb --ExecutePreprocessor.kernel_name=python3 --to notebook --execute /home/ec2-user/SageMaker/TestNotebook.ipynb
echo "Finished nohup command"
conda deactivate
EOF

which I'm trying to use to run the cells in a notebook called TestNotebook.ipynb.

The contents of TestNotebook.ipynb are as follows:

# Cell 1
import pandas as pd
df = pd.DataFrame([1,2,3,7])
df.to_csv('random_junk.csv')

# Cell 2
import boto3

From the error logs and the random_junk.csv file generated, it's clear that cell 1 works fine, but when I uncomment and allow cell 2 in the notebook, I get a complicated error log that includes a lot of badly formatted output. The once legible output in the log that I believe may be relevant is this:

Any help on how to resolve / get around this would be great thanks

[Dan-Treacher]

For anyone encountering the same issue, I appear to have fixed my problem by changing the lifecycle configuration to the following:

#!/bin/bash
sudo -u ec2-user -i <<'EOF'

source activate python3
echo "Installing boto3"
pip install boto3
echo "Running notebook"
jupyter nbconvert --to html --ExecutePreprocessor.kernel_name=python3 --execute /home/ec2-user/SageMaker/TestNotebook.ipynb
echo "Finished running notebook"
conda deactivate
EOF

Changes: altered --to notebook to --to html and removed nohup. I'll update once I've waited the 10 minuted to figure out which one was at fault

[Dan-Treacher]

It was the --to notebook. Including the nohup keyword prior to the jupyter ... line did not break it