aws-samples / amazon-kendra-langchain-extensions

Samples to build Generative AI applications with LangChain and Amazon Kendra
https://aws.amazon.com/blogs/machine-learning/quickly-build-high-accuracy-generative-ai-applications-on-enterprise-data-using-amazon-kendra-langchain-and-large-language-models/
MIT No Attribution
158 stars 104 forks source link

Add 'parameters:' key in transform_input function for llama2 #38

Closed johnphilips3 closed 11 months ago

johnphilips3 commented 1 year ago

I deployed this sample with a meta-textgeneration-llama-2-7b-f endpoint generated from Jumpstart and was getting this error from endpoint due to incorrectly formatted input:

ValueError: Error raised by inference endpoint: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (424) from primary with message "{
  "code":424,
  "message":"prediction failure",
  "error":"Input payload must contain a 'inputs' key and optionally a 'parameters' key containing a dictionary of parameters."

This change adds a 'parameters:' key to the model_kwargs value and corrects the error.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

MithilShah commented 11 months ago

Llama2 7B works differently from 70B. will have to add 7B as a new feature.