leondz / garak

LLM vulnerability scanner
https://discord.gg/uVch4puUCs
Apache License 2.0
1.04k stars 121 forks source link

Split NVCF generator into completion chat #696

Closed leondz closed 1 month ago

leondz commented 1 month ago

NVCF models support at least two interfaces - Completion and Chat styles. They also sometimes require custom headers. This PR splits the NVCF class into two, one for each interface, providing separate methods for building the payload and parsing the output under each interface style. It also adds code to allow external configuration of the payload fields that the NVCF endpoint is queried with.

leondz commented 1 month ago

ps. i know, i know, sorry about the _config indexing jeffrey

leondz commented 1 month ago

Stray thought as I looked at this, are these services really just OpenAICompatible like NIM? ( This is likely a futures effort thought. )

One would hope so! I like where this is going.