Open mak-rayate opened 3 months ago
@mak-rayate have you been able to solve this issue? I am facing the same problem.
Hello @ahmedselhady , No the error is still open but I have found alternative way (not recommended) to call the model. Instead of deploy model using python, deploy endpoint inside Jumpstart and use it in your code as below : llm_codellama = SagemakerEndpoint( endpoint_name='jumpstart-dft-meta-textgeneration-llama-codellama-7b', <-- paste model endpoint here generated by Jumpstart interface region_name=region, model_kwargs=parameters, endpoint_kwargs={"CustomAttributes": 'accept_eula=true'}, content_handler=content_handler )
Describe the bug I'm trying to deploy meta-textgeneration-llama-codellama-7b jumpstart model endpoints using python. In sage-maker notebook code works fine. But deployed same code in app.py file giving an error of ParamValidationError. Tried --> installed upgraded boto3 and botocore lib.
System Python : 3.9.15 SageMaker Version: 2.214.2 boto3 : Version: 1.34.76 botocore : Version: 1.34.76 streamlit :Version: 1.32.2
To reproduce app.py file code :
import streamlit as st
def main(): st.title('LLAMA Speak')
if name == 'main': main()
Logs ParamValidationError: Parameter validation failed: Unknown parameter in PrimaryContainer: "ModelDataSource", must be one of: ContainerHostname, Image, ImageConfig, Mode, ModelDataUrl, Environment, ModelPackageName, InferenceSpecificationName, MultiModelConfig Traceback: File "/opt/conda/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 542, in _run_script exec(code, module.dict) File "/home/sagemaker-user/natural_to_SQL/app.py", line 36, in
main()
File "/home/sagemaker-user/natural_to_SQL/app.py", line 21, in main
predictor = model1.deploy(
File "/opt/conda/lib/python3.9/site-packages/sagemaker/jumpstart/model.py", line 638, in deploy
predictor = super(JumpStartModel, self).deploy(deploy_kwargs.to_kwargs_dict())
File "/opt/conda/lib/python3.9/site-packages/sagemaker/model.py", line 1610, in deploy
self._create_sagemaker_model(
File "/opt/conda/lib/python3.9/site-packages/sagemaker/jumpstart/model.py", line 467, in _create_sagemaker_model
super(JumpStartModel, self)._create_sagemaker_model(
File "/opt/conda/lib/python3.9/site-packages/sagemaker/model.py", line 914, in _create_sagemaker_model
self.sagemaker_session.create_model(create_model_args)
File "/opt/conda/lib/python3.9/site-packages/sagemaker/session.py", line 3887, in create_model
self._intercept_create_request(create_model_request, submit, self.create_model.name)
File "/opt/conda/lib/python3.9/site-packages/sagemaker/session.py", line 6427, in _intercept_create_request
return create(request)
File "/opt/conda/lib/python3.9/site-packages/sagemaker/session.py", line 3875, in submit
self.sagemaker_client.create_model(**request)
File "/opt/conda/lib/python3.9/site-packages/botocore/client.py", line 565, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/opt/conda/lib/python3.9/site-packages/botocore/client.py", line 974, in _make_api_call
request_dict = self._convert_to_request_dict(
File "/opt/conda/lib/python3.9/site-packages/botocore/client.py", line 1048, in _convert_to_request_dict
request_dict = self._serializer.serialize_to_request(
File "/opt/conda/lib/python3.9/site-packages/botocore/validate.py", line 381, in serialize_to_request
raise ParamValidationError(report=report.generate_report())