prompt-security / ps-fuzz

Make your GenAI Apps Safe & Secure :rocket: Test & harden your system prompt
https://www.prompt.security/fuzzer
MIT License
404 stars 54 forks source link

can run bedrock tests #48

Open vpaturi opened 6 months ago

vpaturi commented 6 months ago

I am happy to find that you have bedrock as a supported provider. But when I am trying to configure the test, it gives the following error: 2024-05-14 19:14:00,899 [WARNING] [interactive_mode.py:158]: Wrong value: 3 validation errors for BedrockChat model_id field required (type=value_error.missing) model extra fields not permitted (type=value_error.extra) temperature extra fields not permitted (type=value_error.extra)

Current configuration ... ┌────────────────────┬───────────────────────────────────────┐ │ Option │ Value │ ├────────────────────┼───────────────────────────────────────┤ │ attack_provider │ bedrock │ │ attack_model │ anthropic.claude-3-haiku-20240307-v1 │ │ target_provider │ bedrock │ │ target_model │ anthropic.claude-3-haiku-20240307-v1 │ │ num_attempts │ 3 │ │ num_threads │ 4 │ │ attack_temperature │ 0.6 │ └────────────────────┴───────────────────────────────────────┘

I didnt see much documentation on what are the parameters to provide here so I guessed these targets and model names are appropriate. Any help on getting this running is appreciated.

I already have my AWS cli access with appropriate keys setup as environment variables.