aws / sagemaker-python-sdk

A library for training and deploying machine learning models on Amazon SageMaker
https://sagemaker.readthedocs.io/
Apache License 2.0
2.09k stars 1.13k forks source link

Cannot Deploy Endpoint #2617

Closed ujjawalmadan closed 8 months ago

ujjawalmadan commented 3 years ago

I am not able to deploy the endpoint and am met with this error message. Have tried diagnosing but not sure what the issue is:

import sagemaker

sess = sagemaker.Session()
role = sagemaker.get_execution_role()

from sagemaker.huggingface.model import HuggingFaceModel

# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
   model_data="s3://qfn-transcription/ujjawal_files/model.tar.gz",  # path to your trained sagemaker model
   role=role, # iam role with permissions to create an Endpoint
   transformers_version="4.6", # transformers version used
   pytorch_version="1.7", # pytorch version used
   py_version='py36'
)

# deploy model to SageMaker Inference
predictor = huggingface_model.deploy(
   initial_instance_count=1, 
   instance_type="ml.m5.xlarge"
)

# example request, you always need to define "inputs"
long_text= """
Hugging Face, the winner of VentureBeat’s Innovation in Natural Language Process/Understanding Award for 2021, 
is looking to level the playing field. The team, launched by Clément Delangue and Julien Chaumond in 2016,
was recognized for its work in democratizing NLP, the global market value for which is expected to 
hit $35.1 billion by 2026. This week, Google’s former head of Ethical AI Margaret Mitchell joined the team.
"""

parameters = {'repetition_penalty':4.,'length_penalty': 1.5}

predictor.predict({"inputs":long_text,"parameters":parameters})

# request
predictor.predict(data)

image

mohanasudhan commented 8 months ago

Thank you for bringing up this issue, and I apologize for the delay in the response. Can you confirm that the container and role have sufficient permissions? Additionally, since the issue was raised more than two years ago, can you verify if it still exists? If it has been resolved, could you please provide details on what the issue was and how it was fixed?

mohanasudhan commented 8 months ago

I am closing this issue. Please reopen it if it is still unresolved and provide more details.