leondz / garak

LLM vulnerability scanner
https://discord.gg/uVch4puUCs
Apache License 2.0
1.03k stars 120 forks source link

feature: generator temp / random seed shuffle #745

Open leondz opened 1 week ago

leondz commented 1 week ago

Some models produce the same (or v similar) output every time they're called, regardless of temperature / seed specification. This makes using > 1 generations unproductive.

Add a feature where temperature, or seed, are selected randomly every generation. Config this to disabled by default. It can live in generator-specific config or in _config.run.