leondz
commented
2 weeks ago Yeah, that's my read too. I'm not sure the provider requirement aligns with documented litellm use or the garak generator test as written.
On Thu, Jun 13, 2024, 21:03 Jeffrey Martin @.***> wrote:
@.**** commented on this pull request.
In garak/generators/litellm.py https://github.com/leondz/garak/pull/717#discussion_r1638751939:
@@ -142,6 +155,8 @@ def init(self, name: str = "", generations: int = 10, config_root=_config): " or in the configuration file" )
- self._load_client()
I have never had them run and pass as they both required keys, I had noted in the original PR that it seemed like a config file would be required to instantiate the class. Although there was a comment that said it was not required, the original embedded config parsing https://github.com/leondz/garak/pull/572/files#diff-ff886897d971b8e468ed44457e63890380c3c95e187ee701c2c34e61699bcf44R106-R134 did require a provider. If provider was not found in _config.plugins.generators["litellm.LiteLLMGenerator"] it would raise a ValueError.
I intend to validate function as part of the testing here by setting up a local instance, however there is another issue with this class as the torch_dtype value cannot be accepted as a string. I have fixes for this in progress in the refactor branch I am working one. Short term I was intending to manually patch the torch_dtype default value to allow testing of this change in isolation.
— Reply to this email directly, view it on GitHub https://github.com/leondz/garak/pull/717#discussion_r1638751939, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA5YTVJIDP3EHKQAS7HYUDZHHUJDAVCNFSM6AAAAABIWXJZQ2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDCMJWGY4TQNRYGQ . You are receiving this because you authored the thread.Message ID: @.***>
no need to review/merge until #711 lands