search

aws / amazon-sagemaker-examples

Example 📓 Jupyter notebooks that demonstrate how to build, train, and deploy machine learning models using 🧠 Amazon SageMaker.
https://sagemaker-examples.readthedocs.io
Apache License 2.0
10.18k stars 6.79k forks source link

ClientError: An error occurred (404) when calling the HeadObject operation: Not Found #4479

Open tecena opened 1 year ago

tecena commented 1 year ago

Hello,

I am trying to deploy Huggingface text2text flanT5 model using aws sagemaker. From last one day the model is not getting deployed. facing the below error,

ClientError: An error occurred (404) when calling the HeadObject operation: Not Found

referring to this document and notebook - https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/jumpstart-foundation-models/text2text-generation-flan-t5.ipynb

facing the issue in the step 3 while deploying the model. seems like the there is no model available in the given s3 path, please let us know what is changed.

Thank you

raphsilva commented 1 year ago

I'm getting the same error for https://github.com/aws/amazon-sagemaker-examples/blob/main/introduction_to_amazon_algorithms/jumpstart_text_classification/Amazon_JumpStart_Text_Classification.ipynb

arjun-bali-ds commented 12 months ago

I am also getting the same error while running the model predictor inference piece of code. The error is :

ClientError: An error occurred (ValidationException) when calling the CreateModel operation: Could not find model data at s3://jumpstart-cache-prod-us-east-2/huggingface-text2text/huggingface-text2text-flan-t5-small/artifacts/inference-prepack/v2.0.0/.

for model_id in _MODELCONFIG : endpoint_name = name_from_base(f"jumpstart-example-raglc-{model_id}") inference_instance_type = _MODELCONFIG[model_id]["instance type"]

deploy_image_uri = image_uris.retrieve( region=None, framework=None, # automatically inferred from model_id image_scope="inference", model_id=model_id, model_version=model_version, instance_type=inference_instance_type, )

Retrieve the model uri.

model_uri = model_uris.retrieve( model_id=model_id, model_version=model_version, model_scope="inference" ) model_inference = Model( image_uri=deploy_image_uri, model_data=model_uri, role=aws_role, predictor_cls=Predictor, name=endpoint_name, env=_MODELCONFIG[model_id]["env"], ) model_predictor_inference = model_inference.deploy( initial_instance_count=1, instance_type=inference_instance_type, predictor_cls=Predictor, endpoint_name=endpoint_name, tags=tags ) print(f"{bold}Model {model_id} has been deployed successfully.{unbold}{newline}") _MODELCONFIG[model_id]["endpoint_name"] = endpoint_name

@raphsilva @tecena Did you guys figure any workaround for this ?

rbsteinm commented 7 months ago

Also getting the same error, anyone found a solution for this?

saitaiky commented 5 months ago

Got the same error as @arjun-bali-ds .
ClientError: An error occurred (ValidationException) when calling the CreateModel operation: Could not find model data at s3://jumpstart-cache-prod-us-east-2/huggingface-text2text/huggingface-text2text-flan-t5-small/artifacts/inference-prepack/v2.0.0/.

I have already granted the AmazonS3FullAccess permission to the CDK stack which I use for creating the Sagemaker model

image

Tried to use the policy simular and it can list the jumpstart S3 bucket

image

@raphsilva @tecena @rbsteinm - any luck?