Repository for training and deploying Generative AI models, including text-text, text-to-image generation and prompt engineering playground using SageMaker Studio.
MIT No Attribution
130
stars
88
forks
source link
ModelError for meta-textgeneration-llama-2-7b-f #43
Error :
ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (422) from primary with message "Failed to deserialize the JSON body into the target type: inputs: invalid type: sequence, expected a string at line 1 column 11".
I'm getting an above error for "Multilingual chatbot using E5 multilingual embeddings model and Meta llama2 7-b chat LLM in Sagemaker Jumpstart" notebook
I'm getting an error at section : [Create payload function. Based on the language of the input question ( English, Spanish, or Italian) we are using Amazon Comprehend to idenitfy the language so the prompt for llama2 can be crafted dynamically to ask it to respond in the same language as the question)]
Unable to get where exactly wrong is happening , even after checking payload , it seems fine as below :
{'inputs': [[{'role': 'system',
'content': 'Your are a friendly multilingual Assistant chatbot. You can speak in English, Spanish and Italian. Answer the following QUESTION based only on the CONTEXT given.Respond using the language of the QUESTION and the CONTEXT.If you do not know the answer and the CONTEXT doesn\'t \n contain the answer truthfully say "I don\'t know".\n\n CONTEXT:\n Online features are used in applications required to make real-time predictions. Online features are served from a high-throughput repository with single-digit millisecond latency for fast predictions.\nOffline features are used for training because you need access to very large volumes over a long period of time. These features are served from a high-throughput, high-bandwidth repository.\n\n\n ANSWER:\n '},
{'role': 'user',
'content': 'Respond in English.what are online features?'}]],
'parameters': {'max_new_tokens': 1024,
'top_p': 0.9,
'temperature': 0.1,
'return_full_text': False}}
Error : ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (422) from primary with message "Failed to deserialize the JSON body into the target type: inputs: invalid type: sequence, expected a string at line 1 column 11".
I'm getting an above error for "Multilingual chatbot using E5 multilingual embeddings model and Meta llama2 7-b chat LLM in Sagemaker Jumpstart" notebook
I'm getting an error at section : [Create payload function. Based on the language of the input question ( English, Spanish, or Italian) we are using Amazon Comprehend to idenitfy the language so the prompt for llama2 can be crafted dynamically to ask it to respond in the same language as the question)]
Unable to get where exactly wrong is happening , even after checking payload , it seems fine as below :
{'inputs': [[{'role': 'system', 'content': 'Your are a friendly multilingual Assistant chatbot. You can speak in English, Spanish and Italian. Answer the following QUESTION based only on the CONTEXT given.Respond using the language of the QUESTION and the CONTEXT.If you do not know the answer and the CONTEXT doesn\'t \n contain the answer truthfully say "I don\'t know".\n\n CONTEXT:\n Online features are used in applications required to make real-time predictions. Online features are served from a high-throughput repository with single-digit millisecond latency for fast predictions.\nOffline features are used for training because you need access to very large volumes over a long period of time. These features are served from a high-throughput, high-bandwidth repository.\n\n\n ANSWER:\n '}, {'role': 'user', 'content': 'Respond in English.what are online features?'}]], 'parameters': {'max_new_tokens': 1024, 'top_p': 0.9, 'temperature': 0.1, 'return_full_text': False}}
Any clue ???