[question] how to make generation determinsitic?

turboderp / exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs

MIT License

3.19k stars 234 forks source link

Closed yshui closed 2 weeks ago

yshui commented 1 month ago

I tried using random.seed() to set a fixed seed, but the output still changes.

DocShotgun commented 1 month ago