Portkey-AI / gateway

A Blazing Fast AI Gateway with integrated Guardrails. Route to 200+ LLMs, 50+ AI Guardrails with 1 fast & friendly API.
https://portkey.ai/features/ai-gateway
MIT License
6k stars 410 forks source link

Enhance JSON Schema and JSON Keys plugins with improved error handling and explanations #586

Closed roh26it closed 3 weeks ago

roh26it commented 3 weeks ago

Description

This PR addresses the issues raised in #[Issue Number] by enhancing our JSON Schema and JSON Keys plugins with improved error handling, more detailed explanations, and better JSON extraction capabilities.

Changes Made

  1. Updated both plugins to handle JSON within code blocks and in plain text.
  2. Enhanced error reporting to include detailed explanations of validation results.
  3. Improved JSON extraction to handle multiple JSON objects within a single response.
  4. Updated the plugins to return more informative data objects, including:
    • Matched JSON
    • Present/missing keys (for JSON Keys plugin)
    • Validation errors (for JSON Schema plugin)
    • Explanatory messages for successful and failed validations
  5. Refactored and expanded test suites for both plugins to cover new functionality and edge cases.

How to Test

  1. Run the updated test suites for both plugins.
  2. Manually test the plugins with various input scenarios, including:
    • JSON within code blocks
    • Multiple JSON objects in a single response
    • Invalid JSON
    • Various validation scenarios (e.g., missing keys, schema mismatches)

Expected Behavior

Additional Notes

This update significantly improves the usability and robustness of our JSON validation plugins. Users will now have a better understanding of why their validations succeed or fail, making debugging and troubleshooting much easier.

Checklist

Fixes #585