Mindgard / cli

Test your AI model's security through CLI
https://sandbox.mindgard.ai/
MIT License
14 stars 2 forks source link

implement json output for llm tests #13

Closed admindgard closed 3 months ago

admindgard commented 3 months ago

outputs separately pipe-able (either text or json), similar to curl and other commands. makes usage like the following, where we get the animated output in console and machine format in file. Happy to roll out if we prefer this approach in general.

% mindgard test --json | tee results.json                     
all              ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
attack DevModeV2 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
attack PERSONGPT ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 100% 0:00:00
{
  "id": "<redacted>",
  "displayName": "something",
  ...
  "attacks": [
    {
      "id": "<redacted>",
      "submitted_at": "2024-04-10T17:09:33.352000+00:00",
       ...
    },
  ],
  "isCompleted": true,
  "hasFinished": true,
  "risk": 77,
  "url": "https://sandbox.mindgard.ai/r/test/<redacted>"
}

% cat results.json 
{
  "id": "<redacted>",
  "displayName": "something",
  ...
  "attacks": [
    {
      "id": "<redacted>",
      "submitted_at": "2024-04-10T17:09:33.352000+00:00",
       ...
    },
  ],
  "isCompleted": true,
  "hasFinished": true,
  "risk": 77,
  "url": "https://sandbox.mindgard.ai/r/test/<redacted>"
}