Closed paopa closed 3 months ago
Overall LGTM, I would like to discuss with you the capture_input
part
After a discussion with @cyyeh, we decided not to capture the span input for all steps. Other suggestions, such as the Langfuse debug mode and authentication check, are interesting features. However, we will not include them in this PR as it is focused on the evaluation framework.
This PR aims to integrate the decorator-based Langfuse SDK to collect data for the evaluation process. By default, the environment setup is disabled. To enable Langfuse, follow these steps:
Fill in the environment variables. You can generate the secret key, public key, and host by following the instructions at Langfuse Documentation.
Currently, we don't set the session ID. When implementing the main process for the evaluation, refer to the following code to set up the session ID.
Screenshots