How is long text handled?

michaelfeil / infinity

Infinity is a high-throughput, low-latency REST API for serving text-embeddings, reranking models and clip

https://michaelfeil.github.io/infinity/

MIT License

1.32k stars 97 forks source link

How is long text handled? #88

Closed YanDavKMS closed 8 months ago

YanDavKMS commented 8 months ago

Hey,

I'm trying to understand, what happens if we send a long text, which is longer from the model max length? Will it be truncated by the tokenizer? If not, what happens if the model gets a longer text than it's max length?

Thanks.