microsoft / promptflow

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.
https://microsoft.github.io/promptflow/
MIT License
9.53k stars 872 forks source link

Change eci prefix from upper case to lowercase #3771

Closed diondrapeck closed 2 months ago

diondrapeck commented 2 months ago

Description

This PR changes ECI case from upper to lower to match other evaluator type casing and integrate better with UI evaluation flows.

All Promptflow Contribution checklist:

General Guidelines and Best Practices

Testing Guidelines

github-actions[bot] commented 2 months ago

promptflow-evals test result

 12 files  ±  0   12 suites  ±0   1h 48m 37s :stopwatch: + 1h 31m 28s  64 tests  -  61   61 :white_check_mark:  -  64   3 :zzz: + 3  0 :x: ±0  768 runs   - 732  732 :white_check_mark:  - 768  36 :zzz: +36  0 :x: ±0 

Results for commit 6051601a. ± Comparison against base commit e79da2e2.

This pull request removes 125 and adds 64 tests. Note that renamed tests count towards both. ``` tests.evals.unittests.test_batch_run_context.TestBatchRunContext ‑ test_batch_timeout_custom tests.evals.unittests.test_batch_run_context.TestBatchRunContext ‑ test_batch_timeout_default tests.evals.unittests.test_batch_run_context.TestBatchRunContext ‑ test_with_codeclient tests.evals.unittests.test_batch_run_context.TestBatchRunContext ‑ test_with_pfclient tests.evals.unittests.test_built_in_evaluator.TestBuiltInEvaluators ‑ test_fluency_evaluator tests.evals.unittests.test_built_in_evaluator.TestBuiltInEvaluators ‑ test_fluency_evaluator_empty_string tests.evals.unittests.test_built_in_evaluator.TestBuiltInEvaluators ‑ test_fluency_evaluator_non_string_inputs tests.evals.unittests.test_chat_evaluator.TestChatEvaluator ‑ test_conversation_validation_invalid_citations tests.evals.unittests.test_chat_evaluator.TestChatEvaluator ‑ test_conversation_validation_missing_role tests.evals.unittests.test_chat_evaluator.TestChatEvaluator ‑ test_conversation_validation_normal … ``` ``` tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_chat[False-True] tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_chat[True-True] tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_content_safety tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_content_safety_chat[False-False] tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_content_safety_chat[True-False] tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_qa[False] tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_qa[True] tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_qa_for_nans tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_qa_with_openai_config[False] tests.evals.e2etests.test_builtin_evaluators.TestBuiltInEvaluators ‑ test_composite_evaluator_qa_with_openai_config[True] … ```

:recycle: This comment has been updated with latest results.