aws / sagemaker-python-sdk

A library for training and deploying machine learning models on Amazon SageMaker
https://sagemaker.readthedocs.io/
Apache License 2.0
2.09k stars 1.13k forks source link

Unknown parameter in PrimaryContainer: "ModelDataSource" with older boto version #4321

Open tleyden opened 8 months ago

tleyden commented 8 months ago

Describe the bug When using an older version of the boto library, it will throw the following error:

ParamValidationError: Parameter validation failed:
Unknown parameter in PrimaryContainer: "ModelDataSource", must be one of: ContainerHostname, Image, ImageConfig, Mode, ModelDataUrl, Environment, ModelPackageName, InferenceSpecificationName, MultiModelConfig
Full stack trace ``` Traceback (most recent call last): File "/var/task/app.py", line 90, in handler llm_model.deploy( File "/var/lang/lib/python3.8/site-packages/sagemaker/huggingface/model.py", line 315, in deploy return super(HuggingFaceModel, self).deploy( File "/var/lang/lib/python3.8/site-packages/sagemaker/model.py", line 1601, in deploy self._create_sagemaker_model( File "/var/lang/lib/python3.8/site-packages/sagemaker/model.py", line 907, in _create_sagemaker_model self.sagemaker_session.create_model(**create_model_args) File "/var/lang/lib/python3.8/site-packages/sagemaker/session.py", line 3667, in create_model self._intercept_create_request(create_model_request, submit, self.create_model.__name__) File "/var/lang/lib/python3.8/site-packages/sagemaker/session.py", line 6102, in _intercept_create_request return create(request) File "/var/lang/lib/python3.8/site-packages/sagemaker/session.py", line 3655, in submit self.sagemaker_client.create_model(**request) File "/var/runtime/botocore/client.py", line 530, in _api_call return self._make_api_call(operation_name, kwargs) File "/var/runtime/botocore/client.py", line 919, in _make_api_call request_dict = self._convert_to_request_dict( File "/var/runtime/botocore/client.py", line 990, in _convert_to_request_dict request_dict = self._serializer.serialize_to_request( File "/var/runtime/botocore/validate.py", line 381, in serialize_to_request raise ParamValidationError(report=report.generate_report()) botocore.exceptions.ParamValidationError: Parameter validation failed: Unknown parameter in PrimaryContainer: "ModelDataSource", must be one of: ContainerHostname, Image, ImageConfig, Mode, ModelDataUrl, Environment, ModelPackageName, InferenceSpecificationName, MultiModelConfig ```

Sorry, I don't have the boto version handy but will find it and update the ticket. It was very old.

To reproduce

from sagemaker.huggingface.model import HuggingFaceModel

# create Hugging Face Model Class
huggingface_model = HuggingFaceModel(
   model_data={'S3DataSource':{'S3Uri': "s3://tmybuckaet",'S3DataType': 'S3Prefix','CompressionType': 'None'}},
   role=role,                      # iam role with permissions to create an Endpoint
   transformers_version="4.34.1",  # transformers version used
   pytorch_version="1.13.1",       # pytorch version used
   py_version='py310',              # python version used
   model_server_workers=1,         # number of workers for the model server
)

# Let SageMaker know that we've already compiled the model
huggingface_model._is_compiled_model = True

# deploy the endpoint endpoint
predictor = huggingface_model.deploy(
    initial_instance_count=1,      # number of instances
    instance_type="ml.inf2.xlarge", # AWS Inferentia Instance
    volume_size = 100
)

Expected behavior Deploy without errors

Screenshots or logs Stacktrace above

System information

A description of your system. Please provide:

Additional context

I think sagemaker-python-sdk should require a recent version of boto to avoid this error. The error is very unhelpful since it doesn't give any hint as to the root cause.

martinRenou commented 8 months ago

Sorry, I don't have the boto version handy but will find it and update the ticket. It was very old.

Thank you! We can probably update the boto version requirements when you have more information about this. Otherwise the piece of information in this ticket could be enough for other users who may end up here because they had the same error.