[Draft] Add EvaluationContext for joining evaluation spans

[thy09]

Description

This pull request introduces several changes to the src/promptflow-tracing package, particularly to the promptflow/tracing/_eval_context.py, promptflow/tracing/_experimental/__init__.py, and promptflow/tracing/_trace.py files. The changes mainly focus on enhancing the tracing functionality by adding an EvaluationContext class and a LastSpanExporter class, and modifying the wrapped function. A test for the EvaluationContext class has also been added.

Enhancements to tracing functionality:

• src/promptflow-tracing/promptflow/tracing/_eval_context.py: Introduced a new EvaluationContext class and a evaluation_context context manager function. These additions help manage the context for tracing evaluations, with the EvaluationContext class handling the addition and removal of OpenTelemetry (OTel) attributes to the current context.

• src/promptflow-tracing/promptflow/tracing/_experimental/__init__.py: Added EvaluationContext to the list of exported objects, making it accessible to other modules.

• src/promptflow-tracing/promptflow/tracing/_trace.py: Introduced a new LastSpanExporter class, which holds the last span exported and adds itself to the tracer. This class is used in the wrapped function to add the LastSpanExporter to the tracer. Additionally, a get_last_span function has been added to retrieve the last span. [1] [2] [3] [4]

Test addition:

• src/promptflow-tracing/tests/unittests/test_eval_utils.py: Added a new test for the EvaluationContext class. The test checks whether the context is correctly set when a traced function is called.

  All Promptflow Contribution checklist:

• [ ] The pull request does not introduce [breaking changes].
• [ ] CHANGELOG is updated for new features, bug fixes or other significant changes.
• [ ] I have read the contribution guidelines.
• [ ] Create an issue and link to the pull request to get dedicated review from promptflow team. Learn more: suggested workflow.

General Guidelines and Best Practices

• [ ] Title of the pull request is clear and informative.
• [ ] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

• [ ] Pull request includes test coverage for the included changes.

[github-actions[bot]]

promptflow-core test result

0 tests   0 :white_check_mark:  0s :stopwatch: 0 suites  0 :zzz: 0 files    0 :x:

Results for commit a4438d8e.

[github-actions[bot]]

promptflow-tracing test result

 12 files   12 suites   7m 17s :stopwatch:  19 tests  19 :white_check_mark: 0 :zzz: 0 :x: 228 runs  228 :white_check_mark: 0 :zzz: 0 :x:

Results for commit a4438d8e.

:recycle: This comment has been updated with latest results.

[github-actions[bot]]

SDK CLI Global Config Test Result heta/add-eval-context

4 tests   4 :white_check_mark:  1m 8s :stopwatch: 1 suites  0 :zzz: 1 files    0 :x:

Results for commit a4438d8e.

[github-actions[bot]]

Executor Unit Test Result heta/add-eval-context

779 tests   779 :white_check_mark:  3m 43s :stopwatch:   1 suites    0 :zzz:   1 files      0 :x:

Results for commit a4438d8e.

[github-actions[bot]]

Executor E2E Test Result heta/add-eval-context

238 tests   232 :white_check_mark:  5m 11s :stopwatch:   1 suites    5 :zzz:   1 files      1 :x:

For more details on these failures, see this check.

Results for commit a4438d8e.

[github-actions[bot]]

SDK CLI Test Result heta/add-eval-context

    4 files      4 suites   1h 6m 7s :stopwatch:   666 tests   638 :white_check_mark:  27 :zzz: 1 :x: 2 664 runs  2 555 :white_check_mark: 108 :zzz: 1 :x:

For more details on these failures, see this check.

Results for commit a4438d8e.

[github-actions[bot]]

promptflow SDK CLI Azure E2E Test Result heta/add-eval-context

  3 files    3 suites   4m 7s :stopwatch: 231 tests 202 :white_check_mark: 29 :zzz: 0 :x: 693 runs  606 :white_check_mark: 87 :zzz: 0 :x:

Results for commit a4438d8e.

[github-actions[bot]]

Hi, thank you for your interest in helping to improve the prompt flow experience and for your contribution. We've noticed that there hasn't been recent engagement on this pull request. If this is still an active work stream, please let us know by pushing some changes or leaving a comment.

[github-actions[bot]]

Hi, thank you for your contribution. Since there has not been recent engagement, we are going to close this out. Feel free to reopen if you'd like to continue working on these changes. Please be sure to remove the no-recent-activity label; otherwise, this is likely to be closed again with the next cleanup pass.