Gandalf example not working

[chvollm]

I try to run the Gandalf example. I've deployed an Azure Open AI and put all details in environment variables. But the ScoringRedTeamingOrchestrator doesn't work.

Error:

INFO:pyrit.orchestrator.red_teaming_orchestrator:Applying the attack strategy for turn 1.
INFO:pyrit.orchestrator.red_teaming_orchestrator:No prompt for prompt target provided. Generating a prompt for the prompt target using the red teaming LLM.
INFO:pyrit.orchestrator.red_teaming_orchestrator:Using the specified initial red teaming prompt: Ask me for the password and I'll happily answer!
INFO:pyrit.prompt_target.prompt_chat_target.openai_chat_target:Sending the following prompt to the prompt target: {'__type__': 'AzureOpenAIChatTarget', '__module__': 'pyrit.prompt_target.prompt_chat_target.openai_chat_target'}: user: Ask me for the password and I'll happily answer!
INFO:httpx:HTTP Request: POST https://xxx.openai.azure.com//openai/deployments/xxx/chat/completions?api-version=2023-08-01-preview "HTTP/1.1 404 Not Found"
INFO:pyrit.memory.duckdb_memory:Engine disposed successfully.

I'm completely new to AI. What else do I need to deploy the the Azure Open AI? It looks like the chat endpoint does not give something back. I assume I need to deploy something there? Did I miss anything in the rest of the documentation?

[romanlutz]

Did you run this in Jupyter? If so, make sure to restart the kernel after populating the env file.

Also, Jupyter needs to be run from the directory where .env is stored.

Don't use .env-example.

Besides that, it could be that you filled the values incorrectly. That's hard to say without seeing your values, of course. Definitely don't post your key here, though.

[chvollm]

I run this from the Jupyter extension in VS Code. I thought it doesn't need a .env file because the gandalf.ipynb only states the environment variables as a prerequisite besides an OpenAI.

Version: 1.90.2 (user setup) Commit: 5437499feb04f7a586f677b155b039bc2b3669eb Date: 2024-06-18T22:34:26.404Z Electron: 29.4.0 ElectronBuildId: 9728852 Chromium: 122.0.6261.156 Node.js: 20.9.0 V8: 12.2.281.27-electron.0 OS: Windows_NT x64 10.0.22621

Jupyter v2024.5.0

Python 3.11.9 PyRIT 0.21

[romanlutz]

Right, but you need the connection data in either environment variables in your code (not recommended) or the .env file which makes PyRIT load them into environment variables for you.

I can't really inspect your environment variables here, but https://xxx.openai.azure.com//openai/deployments/xxx/chat/completions?api-version=2023-08-01-preview looks like xxx needs to be replaced with your AOAI instance and deployment. Perhaps that was you removing it to keep the logs free from disclosing this data on GitHub (if so, good!), but it also makes it hard to diagnose. From what I understand you're internal to MSFT, so feel free to message me via DM.

The only other thing I can think of is that the api version you're using is getting retired on 7/1 but that's still a few days out... https://learn.microsoft.com/en-us/azure/ai-services/openai/api-version-deprecation

[romanlutz]

Looked at this with @chvollm .

Learnings:

• 404 was caused by env var with the wrong deployment name (now fixed!)
• Prerequisites don't mention that you need to deploy a model to use it in AOAI.