Infinite token generation

marella / ctransformers

Python bindings for the Transformer models implemented in C/C++ using GGML library.

MIT License

1.76k stars 137 forks source link

Closed yukiarimo closed 6 months ago

yukiarimo commented 7 months ago

Is there any way to make an infinite token generation? Is there an ability to set the maximum new tokens to infinity or any other workaround?

yukiarimo commented 6 months ago

It seems impossible, so I'm going to use the while True loop over it.