search

leondz / garak

LLM vulnerability scanner
https://discord.gg/uVch4puUCs
Apache License 2.0
1.03k stars 120 forks source link

give more friendly message when `openaicompatible`/`nim` named model not found #724

Open leondz opened 3 weeks ago

leondz commented 3 weeks ago

example trace with invalid name:

    outputs = self._call_model(prompt, 1)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lderczynski/dev/garak/garak/generators/nim.py", line 66, in _call_model
    return super()._call_model(prompt, generations_this_call)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lderczynski/anaconda3/envs/garak/lib/python3.11/site-packages/backoff/_sync.py", line 105, in retry
    ret = target(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lderczynski/dev/garak/garak/generators/openai.py", line 196, in _call_model
    response = self.generator.create(
               ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lderczynski/anaconda3/envs/garak/lib/python3.11/site-packages/openai/_utils/_utils.py", line 277, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/lderczynski/anaconda3/envs/garak/lib/python3.11/site-packages/openai/resources/chat/completions.py", line 590, in create
    return self._post(
           ^^^^^^^^^^^
  File "/home/lderczynski/anaconda3/envs/garak/lib/python3.11/site-packages/openai/_base_client.py", line 1240, in post
    return cast(ResponseT, self.request(cast_to, opts, stream=stream, stream_cls=stream_cls))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lderczynski/anaconda3/envs/garak/lib/python3.11/site-packages/openai/_base_client.py", line 921, in request
    return self._request(
           ^^^^^^^^^^^^^^
  File "/home/lderczynski/anaconda3/envs/garak/lib/python3.11/site-packages/openai/_base_client.py", line 1020, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.NotFoundError: 404 page not found