langfuse / langfuse-docs

🪢 Langfuse documentation -- Langfuse is the open source LLM Engineering Platform. Observability, evals, prompt management, playground and metrics to debug and improve LLM apps
https://langfuse.com
MIT License
53 stars 52 forks source link

Updated Cookbook: Example for Fetching Scores from Langfuse #857

Open Sohammhatre10 opened 2 days ago

Sohammhatre10 commented 2 days ago

Description

This update provides an example of using the fetch_scores() function from Langfuse to retrieve evaluation metrics. The example integrates UpTrain and Ragas for model evaluation and demonstrates how to log and fetch scores within Langfuse as mentioned in https://github.com/langfuse/langfuse/issues/3505

Key Features

  1. Evaluation with UpTrain and Ragas:

    • Provides examples for evaluating context relevance, factual accuracy, response completeness, context precision, faithfulness, and answer relevancy..
  2. Fetching Scores:

    • Shows how to retrieve and filter scores using fetch_scores_from_langfuse.
  3. Correlation Matrix Visualization:

    • Adds a section that calculates and visualizes the correlation between UpTrain and Ragas evaluation scores using a heatmap.

[!IMPORTANT] Adds an example for using Langfuse to fetch scores, evaluate models with UpTrain and Ragas, and visualize results using a correlation matrix.

  • Behavior:
    • Adds example for using fetch_scores() from Langfuse to retrieve evaluation metrics.
    • Demonstrates integration with UpTrain and Ragas for model evaluation.
    • Shows how to log and fetch scores within Langfuse.
  • Visualization:
    • Includes a section for calculating and visualizing correlation between evaluation scores using a heatmap.
  • Misc:
    • Minor whitespace changes in dspy.md, instructor.md, example-javascript.md, example-python-langgraph.md, example-python-instrumentation-module.md, example-python.md, example-vercel-ai.md, example_external_evaluation_pipelines.md, integration_dspy.md, integration_instructor.md, integration_langgraph.md, integration_llama-index_instrumentation.md, integration_llama_index_posthog_mistral.md, integration_mirascope.md, integration_mistral_sdk.md, integration_ollama.md, integration_openai_structured_output.md, example-langchain.md, js_integration_langchain.md, js_tracing_example_vercel_ai_sdk.md, prompt_management_langchain.md.

This description was created by Ellipsis for 02ebc242f05aa872fb1742d8b1514c5dd217a0aa. It will automatically update as commits are pushed.

review-notebook-app[bot] commented 2 days ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

vercel[bot] commented 2 days ago

@Sohammhatre10 is attempting to deploy a commit to the langfuse Team on Vercel.

A member of the Team first needs to authorize it.

CLAassistant commented 2 days ago

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
0 out of 2 committers have signed the CLA.

:x: Your Name
:x: Sohammhatre10


Your Name seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.