search

huggingface / text-embeddings-inference

A blazing fast inference solution for text embeddings models
https://huggingface.co/docs/text-embeddings-inference/quick_tour
Apache License 2.0
2.59k stars 161 forks source link

Input validation error: inputs must have less than 512 tokens #369

Open ellahe-git opened 1 month ago

ellahe-git commented 1 month ago

text-embeddings-router start an embedding serving, but always got the error 【 Input validation error: inputs must have less than 512 tokens】，which param should i use to change max tokens of input？512 is too short, i can not find true param in RAEDME or --help

To solve the issue of 'Input validation error: inputs must have less than 512 tokens', I added the auto truncate parameter to' true 'when starting the Docker image, but the image failed to start. What is the reason for this? The startup command is as follows: Docker run -- rm -- gpus all - d - p 18082:80-- name multilingual-e5-large-v 'model mount address'' mirror address' -- model id 'model id' --auto-truncate true

vjsrinath commented 1 month ago

I am also facing the same issue. Looks like --max-input-tokens flag has not been written to allow us to specify it during the startup.

lluck42 commented 3 weeks ago

I am also facing the same issue, too

atlas-comstock commented 2 weeks ago

same here

lluck42 commented 2 weeks ago

this is a problem of model but not TEI. You can see on other issue. issue365