search

huggingface / chat-ui

Open source codebase powering the HuggingChat app
https://huggingface.co/chat
Apache License 2.0
6.82k stars 964 forks source link

Tokens limits issue #327

Open Billyroot opened 1 year ago

Billyroot commented 1 year ago

Input validation error: inputs tokens + max_new_tokens must be <= 1512. Given: 603 inputs tokens and 1024max_new_tokens

When deployed, the ui is working fine for like 2 or 3 promts, then every prompt we try we get a red line on top with a pop-up having this message. Please how can we remove this limitation on the code?

nsarrazin commented 1 year ago

Hi! Can I see your model definition in your .env ? Are you hosting your own endpoint ? :eyes:

Billyroot commented 1 year ago

Hi! Can I see your model definition in your .env ? Are you hosting your own endpoint ? :eyes:

Thanks for the reply.

We used Pythia epoch 3.5 and guanaco merged. We are relying on hugging face api inference,I don't know if it replies to your question about endpoints.

nsarrazin commented 10 months ago

Hi, sorry for the delay, is this issue still relevant ? Let me know if so!