search

Azure-Samples / ai-rag-chat-evaluator

Tools for evaluation of RAG Chat Apps using Azure AI Evaluate SDK and OpenAI
MIT License
163 stars 59 forks source link

Can the generate script create the output dir/path if it doesn't exist? #9

Closed dfberry closed 5 months ago

dfberry commented 5 months ago 
2024-01-12 23:51:12 (INFO) scripts: Writing 16 questions to /workspaces/ai-rag-chat-evaluator/example_input_2/qa.jsonl
Traceback (most recent call last):

  File "<frozen runpy>", line 198, in _run_module_as_main

  File "<frozen runpy>", line 88, in _run_code

  File "/workspaces/ai-rag-chat-evaluator/scripts/__main__.py", line 6, in <module>
    app()

  File "/workspaces/ai-rag-chat-evaluator/scripts/cli.py", line 36, in generate
    generate_test_qa_data(

  File "/workspaces/ai-rag-chat-evaluator/scripts/generate.py", line 47, in generate_test_qa_data
    with open(output_file, "w") as f:
         ^^^^^^^^^^^^^^^^^^^^^^

FileNotFoundError: [Errno 2] No such file or directory: '/workspaces/ai-rag-chat-evaluator/example_input_2/qa.jsonl'
pamelafox commented 5 months ago

Huh I thought it did but I admit being surprised that it did. I'll look into that, thanks.

pamelafox commented 5 months ago

This is now supported, thanks for filing.