Pytorch deployment expects string for image_uri but still gives TypeError

Describe the bug I am using PytorchModel to deploy a sagemaker endpoint. I define it using PytorchModel() and I feed in a string for image_uri and leave framework_version and py_version blank. Since this seems to be the suggested steps to take as shown:

https://sagemaker.readthedocs.io/en/v1.16.3/sagemaker.pytorch.html#sagemaker.pytorch.model.PyTorchModel

To reproduce Start new sagemaker kernel with conda_pytorch_latest_py36 and execute the code.

import sagemaker
import os
os.environ['AWS_DEFAULT_REGION'] = 'us-east-1'
role = sagemaker.get_execution_role()

from sagemaker.pytorch import PyTorchModel

model_data = 's3://bucket/path/to/my/folder/model_data.tar.gz'
entry_point = 'deploy_ei.py'
source_dir = 's3://bucket/path/to/my/folder/deploy.tar.gz'

#Different image_uri i found online to try:

#Images I found using public ECR images: https://gallery.ecr.aws/
# image_uri = 'public.ecr.aws/amazonlinux/amazonlinux:latest'
# image_uri = 'public.ecr.aws/lambda/python:latest'
# image_uri = 'public.ecr.aws/bitnami/pytorch:latest'

#Different Pytorch ECR images I found from https://docs.aws.amazon.com/sagemaker/latest/dg/neo-deployment-hosting-services-container-images.html
#or from other AWS websites
# image_uri = '785573368785.dkr.ecr.us-east-1.amazonaws.com/sagemaker-inference-pytorch:1.6-py3'

#Pytorch Docker image I found from https://hub.docker.com/r/pytorch/pytorch/tags?page=1&ordering=last_updated
# image_uri = 'pytorch/pytorch:1.7.0-cuda11.0-cudnn8-runtime'
pytorch = PyTorchModel(
    model_data=model_data,
    role=role,
    entry_point=entry_point,
    source_dir=source_dir,
    image_uri=image_uri)
#     framework_version='1.3.1',
#     py_version='py3')

#DEPLOYING SAGEMAKER
from sagemaker.serializers import JSONSerializer
from sagemaker.deserializers import JSONDeserializer

local_mode=False

if local_mode:
    instance_type='local'
    accelerator_type='local_sagemaker_notebook'
else:
#     instance_type='ml.c4.xlarge'
    instance_type = 'ml.r5.xlarge'
    accelerator_type='ml.eia2.xlarge'

predictor = pytorch.deploy(
    initial_instance_count=1,
    instance_type=instance_type,
    accelerator_type=accelerator_type,
    serializer=JSONSerializer(),
    deserializer=JSONDeserializer()
    )

MY PIP ENVIRONMENT:

My pip environment:

Package                            Version
---------------------------------- -------------------
aiobotocore                        1.2.1
aiohttp                            3.7.3
aioitertools                       0.7.1
alabaster                          0.7.12
anaconda-client                    1.7.2
anaconda-project                   0.9.1
anyio                              2.1.0
appdirs                            1.4.4
argh                               0.26.2
argon2-cffi                        20.1.0
asn1crypto                         1.4.0
astroid                            2.5
astropy                            4.1
async-generator                    1.10
async-timeout                      3.0.1
atomicwrites                       1.4.0
attrs                              20.3.0
autopep8                           1.5.5
autovizwidget                      0.18.0
awscli                             1.19.34
Babel                              2.9.0
backcall                           0.2.0
backports.functools-lru-cache      1.6.1
backports.os                       0.1.1
backports.shutil-get-terminal-size 1.0.0
bcrypt                             3.2.0
beautifulsoup4                     4.9.3
bitarray                           1.6.3
bkcharts                           0.2
black                              20.8b1
bleach                             3.3.0
bokeh                              2.2.3
boto                               2.49.0
boto3                              1.17.34
botocore                           1.20.34
Bottleneck                         1.3.2
brotlipy                           0.7.0
cached-property                    1.5.1
certifi                            2020.12.5
cffi                               1.14.5
chardet                            3.0.4
click                              7.1.2
cloudpickle                        1.6.0
clyent                             1.2.2
colorama                           0.4.3
contextlib2                        0.6.0.post1
contextvars                        2.4
coverage                           5.4
cryptography                       3.4.4
cycler                             0.10.0
Cython                             0.29.22
cytoolz                            0.11.0
dask                               2021.2.0
dataclasses                        0.8
decorator                          4.4.2
defusedxml                         0.6.0
diff-match-patch                   20200713
dill                               0.3.3
diskcache                          5.2.1
distributed                        2021.2.0
distro                             1.5.0
docker                             4.4.4
docker-compose                     1.28.6
dockerpty                          0.4.1
docopt                             0.6.2
docutils                           0.15.2
entrypoints                        0.3
environment-kernels                1.1.1
et-xmlfile                         1.0.1
fastcache                          1.1.0
flake8                             3.8.4
Flask                              1.1.2
fsspec                             0.8.7
future                             0.18.2
gevent                             21.1.2
gmpy2                              2.1.0b1
google-pasta                       0.2.0
greenlet                           0.4.17
h5py                               3.1.0
hdijupyterutils                    0.18.0
HeapDict                           1.0.1
helpdev                            0.7.1
html5lib                           1.1
hypothesis                         6.3.2
idna                               2.10
idna-ssl                           1.1.0
imageio                            2.9.0
imagesize                          1.2.0
immutables                         0.15
importlib-metadata                 3.7.0
iniconfig                          1.1.1
intervaltree                       3.0.2
ipykernel                          5.5.0
ipyparallel                        6.3.0
ipython                            7.16.1
ipython-genutils                   0.2.0
ipywidgets                         7.6.3
isort                              5.7.0
itsdangerous                       1.1.0
jdcal                              1.4.1
jedi                               0.17.2
jeepney                            0.6.0
Jinja2                             2.11.3
jmespath                           0.10.0
joblib                             1.0.1
json5                              0.9.5
jsonschema                         3.2.0
jupyter                            1.0.0
jupyter-client                     6.1.11
jupyter-console                    6.2.0
jupyter-core                       4.7.1
jupyter-packaging                  0.7.12
jupyter-server                     1.4.1
jupyterlab                         3.0.9
jupyterlab-pygments                0.1.2
jupyterlab-server                  2.3.0
jupyterlab-widgets                 1.0.0
keyring                            22.0.1
kiwisolver                         1.3.1
lazy-object-proxy                  1.5.2
llvmlite                           0.35.0
locket                             0.2.0
lxml                               4.4.1
MarkupSafe                         1.1.1
matplotlib                         3.3.4
mccabe                             0.6.1
mistune                            0.8.4
mkl-fft                            1.2.1
mkl-random                         1.2.0
mkl-service                        2.3.0
mock                               4.0.3
more-itertools                     8.7.0
mpmath                             1.2.1
msgpack                            1.0.2
multidict                          5.1.0
multipledispatch                   0.6.0
mypy-extensions                    0.4.3
nb-conda                           2.2.1
nb-conda-kernels                   2.3.1
nbclassic                          0.2.6
nbclient                           0.5.2
nbconvert                          6.0.7
nbformat                           5.1.2
nest-asyncio                       1.4.3
networkx                           2.5
nltk                               3.4.4
nose                               1.3.7
notebook                           6.2.0
numba                              0.52.0
numexpr                            2.7.2
numpy                              1.19.2
numpydoc                           1.1.0
olefile                            0.46
onnx                               1.5.0
opencv-python                      4.5.1.48
openpyxl                           3.0.6
packaging                          20.9
pandas                             1.1.5
pandocfilters                      1.4.2
paramiko                           2.7.2
parso                              0.7.0
partd                              1.1.0
path                               15.1.2
pathlib2                           2.3.5
pathspec                           0.8.1
patsy                              0.5.1
pep8                               1.7.1
petastorm                          0.9.8
pexpect                            4.8.0
pickleshare                        0.7.5
Pillow                             8.1.0
pip                                21.0.1
plotly                             4.14.3
pluggy                             0.13.1
ply                                3.11
prometheus-client                  0.9.0
prompt-toolkit                     3.0.5
protobuf                           3.15.5
protobuf3-to-dict                  0.1.5
psutil                             5.8.0
psycopg2                           2.7.5
ptyprocess                         0.7.0
py                                 1.10.0
py4j                               0.10.7
pyarrow                            3.0.0
pyasn1                             0.4.8
pycodestyle                        2.6.0
pycosat                            0.6.3
pycparser                          2.20
pycrypto                           2.6.1
pycurl                             7.43.0.6
pydocstyle                         5.1.1
pyflakes                           2.2.0
pyfunctional                       1.4.3
pygal                              2.4.0
Pygments                           2.8.0
pykerberos                         1.2.1
pylint                             2.7.0
pyls-black                         0.4.6
pyls-spyder                        0.3.2
PyNaCl                             1.4.0
pyodbc                             4.0.30
pyOpenSSL                          20.0.1
pyparsing                          2.4.7
pyrsistent                         0.17.3
PySocks                            1.7.1
pyspark                            2.3.4
pytest                             6.2.2
pytest-arraydiff                   0.3
pytest-astropy                     0.8.0
pytest-astropy-header              0.1.2
pytest-cov                         2.11.1
pytest-doctestplus                 0.9.0
pytest-filter-subpackage           0.1.1
pytest-mock                        3.5.1
pytest-openfiles                   0.5.0
pytest-remotedata                  0.3.2
python-dateutil                    2.8.1
python-dotenv                      0.15.0
python-jsonrpc-server              0.4.0
python-language-server             0.36.2
pytz                               2021.1
PyWavelets                         1.1.1
pyxdg                              0.26
PyYAML                             5.4.1
pyzmq                              22.0.3
QDarkStyle                         2.8.1
QtAwesome                          1.0.2
qtconsole                          5.0.2
QtPy                               1.9.0
regex                              2020.11.13
requests                           2.25.1
requests-kerberos                  0.12.0
retrying                           1.3.3
rope                               0.18.0
rsa                                4.5
Rtree                              0.9.7
ruamel-yaml-conda                  0.15.80
s3fs                               0.4.2
s3transfer                         0.3.4
sagemaker                          2.31.0
sagemaker-pyspark                  1.4.1
scikit-image                       0.16.2
scikit-learn                       0.24.1
scipy                              1.5.3
seaborn                            0.11.1
SecretStorage                      3.3.1
Send2Trash                         1.5.0
setuptools                         49.6.0.post20210108
simplegeneric                      0.8.1
singledispatch                     0.0.0
six                                1.15.0
sklearn                            0.0
smclarify                          0.1
smdebug-rulesconfig                1.0.1
sniffio                            1.2.0
snowballstemmer                    2.1.0
sortedcollections                  2.1.0
sortedcontainers                   2.3.0
soupsieve                          2.0.1
sparkmagic                         0.15.0
Sphinx                             3.5.1
sphinxcontrib-applehelp            1.0.2
sphinxcontrib-devhelp              1.0.2
sphinxcontrib-htmlhelp             1.0.3
sphinxcontrib-jsmath               1.0.1
sphinxcontrib-qthelp               1.0.3
sphinxcontrib-serializinghtml      1.1.4
sphinxcontrib-websupport           1.2.4
spyder                             4.2.0
spyder-kernels                     1.10.0
SQLAlchemy                         1.3.23
statsmodels                        0.12.2
sympy                              1.7.1
tables                             3.6.1
tabulate                           0.8.9
tblib                              1.6.0
terminado                          0.9.2
testpath                           0.4.4
texttable                          1.6.3
threadpoolctl                      2.1.0
three-merge                        0.1.1
toml                               0.10.2
toolz                              0.11.1
torch                              1.7.1
torch-model-archiver               0.2.1
torchserve                         0.2.1
torchvision                        0.8.2
tornado                            6.1
traitlets                          4.3.3
typed-ast                          1.4.2
typing                             3.7.4.3
typing-extensions                  3.7.4.3
ujson                              4.0.2
unicodecsv                         0.14.1
urllib3                            1.26.3
watchdog                           2.0.2
wcwidth                            0.2.5
webencodings                       0.5.1
websocket-client                   0.58.0
Werkzeug                           1.0.1
wheel                              0.36.2
widgetsnbextension                 3.5.1
wrapt                              1.12.1
wurlitzer                          2.0.1
xlrd                               2.0.1
XlsxWriter                         1.3.7
xlwt                               1.3.0
yapf                               0.30.0
yarl                               1.6.3
zict                               2.0.0
zipp                               3.4.0
zope.event                         4.5.0
zope.interface                     5.2.0

Expected behavior I expected this code to execute with no errors. I am passing in a string in the image_uri as requested so I don't understand why it's complaining about type error. I also tried replacing image_uri='GIBBITY GLOBBITY GOO HARRY POTTER' and It returned the same error above- i.e. TypeError.

If I replace PytorchModel() with:

pytorch = PyTorchModel(
    model_data=model_data,
    role=role,
    entry_point=entry_point,
    source_dir=source_dir,
    image_uri=image_uri)
#     framework_version='1.3.1',
#     py_version='py3')

Then it deploys with no problem and can predict correctly. As such, I concluded I may be feeding into image_uri incorrectly?

Screenshots or logs Error when executing code above:

System information A description of your system. Please provide: Amazon Linux AMI 2018.03 ID_LIKE="rhel fedora" I am using the default conda_pytorch_latest_py36 kernel provided by AWS sagemaker notebooks using ml.r5.xlarge instance types

SageMaker Python SDK version: 2.31.0
Framework name (eg. PyTorch) or algorithm (eg. KMeans): PyTorch
Framework version: Pytorch - 1.7.1
Python version: 3.6.13
CPU or GPU: CPU
Custom Docker image (Y/N): N

Additional context

Ultimately, my GOAL IS TO DEPLOY ANY Sagemaker Endpoint that can predict using Pytorch version > 1.5. Here are some other steps I attempted, but don't receive the same bug error above.

I also tried including all 3 (image_uri, framework_version, and py_version) and it seems to work: However, I can't seem to get any image to work:

#Pytorch Docker image I found from https://hub.docker.com/r/pytorch/pytorch/tags?page=1&ordering=last_updated
image_uri = 'pytorch/pytorch:1.7.0-cuda11.0-cudnn8-runtime'

pytorch = PyTorchModel(
    model_data=model_data,
    role=role,
    entry_point=entry_point,
    source_dir=source_dir,
    image_uri=image_uri,
    framework_version='1.3.1',
    py_version='py3')

Returns this error:

#Different Pytorch ECR images I found from https://docs.aws.amazon.com/sagemaker/latest/dg/neo-deployment-hosting-services-container-images.html
image_uri = '785573368785.dkr.ecr.us-east-1.amazonaws.com/sagemaker-inference-pytorch:1.6-py3'

pytorch = PyTorchModel(
    model_data=model_data,
    role=role,
    entry_point=entry_point,
    source_dir=source_dir,
    image_uri=image_uri,
    framework_version='1.3.1',
    py_version='py3')

Returns:

#Images I found using public ECR images: https://gallery.ecr.aws/
image_uri = 'public.ecr.aws/bitnami/pytorch:latest'

pytorch = PyTorchModel(
    model_data=model_data,
    role=role,
    entry_point=entry_point,
    source_dir=source_dir,
    image_uri=image_uri,
    framework_version='1.3.1',
    py_version='py3')

Returns:

aws / sagemaker-python-sdk

Pytorch deployment expects string for image_uri but still gives TypeError #2250