Add "Low-resource" running mode - Githubissues

leondz / garak

LLM vulnerability scanner

https://discord.gg/uVch4puUCs

Apache License 2.0

1.04k stars 121 forks source link

Add "Low-resource" running mode #674

Open leondz opened 1 month ago

leondz commented 1 month ago

Mode for being hyper-conservative with resources

free inference results/objects on CPU
generations=1
avoid detectors/probes that are heavy
assume "resource" refers to RAM, VRAM, CPU cycles (we're not going to get into supporting slow GPUs, this hasn't been a use case emerging from the community)

@erickgalinkin feel free to chime if i missed anything / with extra hints