search

Azure / PyRIT

The Python Risk Identification Tool for generative AI (PyRIT) is an open access automation framework to empower security professionals and machine learning engineers to proactively find risks in their generative AI systems.
MIT License
1.83k stars 355 forks source link

FEAT Export Conversations and Scores in the same JSON file #422

Open nina-msft opened 2 weeks ago

nina-msft commented 2 weeks ago

Is your feature request related to a problem? Please describe.

When conversations are added to memory, they have a specific ID. If Conversations have been scored, the ScoreEntry's prompt_request_response_id directly matches the PromptMemoryEntry id. Currently, memory_interface only has 2 functions to export the database conversations to JSON: export_conversation_by_orchestrator_id and export_conversation_by_id. This is great for getting conversations from the DB (but only for PromptMemoryEntry).

Describe the solution you'd like

Add a function to the memory_interface for DuckDB that exports conversations and their scores into a JSON file.

This may require creation of an intermediary function that exports scores by prompt_request_response_id into a JSON file. The result will be one file that has an entry per Prompt Request Response ID with conversation and score information fields.

nina-msft commented 2 weeks ago

It won't let me assign you Siddhi, but thank you for opening that PR!