protectai / llm-guard

The Security Toolkit for LLM Interactions
https://llm-guard.com/
MIT License
970 stars 111 forks source link

Changing configuration of scanners after initializing with loading models #154

Open nashugame opened 1 week ago

nashugame commented 1 week ago

Hi Guys, I am trying to LLM guard in our group enterprise's solution. The problem we are facing we want to make different paramters of input scanners to changed as per user request dynamically, one way to do it will reinitialize scanner on each user request which will reload the models (associated with different scanner config). But this is not going to scale up. I am looking at something wherein we can initialize scanner only once (load model once with default config) and dynamically change the config of these scanners as per user's request. I want to know if it's at all possible to achieve this and how?

asofter commented 1 week ago

Hey @nashugame , thanks for reaching out. We have received such requests multiple times. However, it will require refactoring of all scanners which we are planning eventually but it's not a priority at a given moment. Yet, we'd be happy to accept Pull Requests with suggestions on how to achieve this.