search

run-llama / llama_index

LlamaIndex is a data framework for your LLM applications
https://docs.llamaindex.ai
MIT License
33.24k stars 4.65k forks source link

[Bug]: Bedrock cohere still not working as expected #14118

Open mgrabmayr opened 2 weeks ago

mgrabmayr commented 2 weeks ago

Bug Description

bug https://github.com/run-llama/llama_index/pull/13442 was resolved. however, I still face issues running cohere.embed-multilingual-v3 via bedrock. Error creating document nodes: An error occurred (ValidationException) when calling the InvokeModel operation: Input is too long for requested model. It fails with 986 characters and below 180 tokens (far below the 2048 and 512 tokens claimed on (https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-embed.html). for smaller quantities, it runs without issues

Version

llama-index-embeddings-bedrock="0.2.1", llama-index="0.10.44"

Steps to Reproduce

node = TextNode( text=chunk_text, metadata=metadata_node, document_id=doc_uuid, ) embed_model = model = BedrockEmbedding( model=model_name, aws_access_key_id=aws_access_key_id, aws_secret_access_key=aws_secret_access_key, region_name=region, callback_manager=callback_manager, ) node.embedding = await embed_model.aget_text_embedding(node.text)

Relevant Logs/Tracbacks

No response

dosubot[bot] commented 2 weeks ago

The "ValidationException: Input is too long for requested model" error occurs because the input text length exceeds the maximum allowed length for the cohere.embed-multilingual-v3 model. The removal of the "Truncate" parameter does not change the inherent limitations of the model regarding input length.

To resolve this, ensure that the input text length is within the acceptable limits for the cohere.embed-multilingual-v3 model. You can manually truncate or split the input text before sending it to the model [1].

To continue talking to Dosu, mention @dosu.