search

promptfoo / promptfoo

Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command line and CI/CD integration.
https://promptfoo.dev
MIT License
4.7k stars 368 forks source link

`outputPath` is not working #1733

Closed albertlieyingadrian closed 1 month ago

albertlieyingadrian commented 1 month ago

Tried setting up outputPath in the config as both string and array of string following this guide https://www.promptfoo.dev/docs/configuration/parameters/#output-file. However, both didn't work 

tests:
  - file://datasets/customer/mini_default_test.yaml

outputPath: 
  - reports/customer_output.html
# outputPath: 'reports/customer.html'

Suspect that it is caused by outputPath is not read in readConfigs function

https://github.com/promptfoo/promptfoo/blob/e6089ec7594282f9be6094a52e8483e044cd7e78/src/config.ts#L357-L390

albertlieyingadrian commented 1 month ago

@mldangelo I went ahead with a PR https://github.com/promptfoo/promptfoo/pull/1734

typpo commented 1 month ago

Shipped in 0.89.4!