Issue: Integration of Langfuse with DSPy Framework - Zero Values for Cost and Token Usage

[mahitha-29]

I am currently integrating Langfuse with the DSPy framework in my project. While I can see the traces in the Langfuse dashboard, the cost and token usage values are always shown as zero for every trace. This issue makes it difficult to track and manage the actual costs and token consumption of my language model usage.

Steps Taken

1. Integrated Langfuse with DSPy as per the documentation.
2. Configured the OpenAI model and ensured the API key is correctly set.
3. Used the observe decorator to capture the traces.
4. Verified that the traces appear in the Langfuse dashboard.

Code Example

import dspy import streamlit as st import os from PyPDF2 import PdfReader from dotenv import load_dotenv from langfuse.decorators import observe

load_dotenv()

api_key = os.getenv("OPENAI_API_KEY")

turbo = dspy.OpenAI( model="gpt-3.5-turbo", api_key=api_key, max_tokens=1000 )

dspy.settings.configure(lm=turbo)

st.set_page_config(page_title="Resume Chatbot") st.title("Resume Chatbot")

with st.sidebar: st.title("Documents") resume = st.file_uploader("Upload the resume", type="pdf")

if resume is not None: extracted_text = "" document = PdfReader(resume) for page in document.pages: extracted_text += page.extract_text()

with st.expander("Extracted resume text"):
    st.write(extracted_text)

class GenerateQuestions(dspy.Signature):
    """
   Generate  interview questions based on the resume in points.
    """
     resume_text = dspy.InputField(desc="Candidate resume details")
    questions = dspy.OutputField(desc="Interview questions related to the resume")

@observe()
def get_questions(resume):
    generator = dspy.ChainOfThought(GenerateQuestions)
    output = generator(resume_text=resume)
    return output

result = get_questions(resume=extracted_text)

st.write("AI Generated Output:")
st.markdown(result)

I would appreciate any guidance or suggestions to resolve this issue. Specifically,

Are there any additional configurations required to accurately capture and display cost and token usage?

[okhat]

Thank you @mahitha-29. Can you tag the langfuse team, in particular whoever made the contribution to DSPy? You can find this from the git commit history.

[arnavsinghvi11]

Also tagging related merged Langfuse PR #1186 if this helps

[mahitha-29]

Also tagging related merged Langfuse PR #1186 if this helps

Yeah, thats helps. Thank you very much. @arnavsinghvi11 Do we need to clone dspy repo for using this feature of integration with langfuse?

[arnavsinghvi11]

Attached is some relevant documentation on DSPy-Langfuse. you can continue to configure DSPy LMs in the same way, and just do .tracker_call to use Langfuse. lmk if that helps!

[mahitha-29]

Attached is some relevant documentation on DSPy-Langfuse. you can continue to configure DSPy LMs in the same way, and just do .tracker_call to use Langfuse. lmk if that helps!

@arnavsinghvi11 @xucailiang Could you help in clarifying this? As per the latest DSPy v2.4.13 release, which was done three weeks ago, it doesn't cover the integration of Langfuse with DSPy. If we use the standard DSPy library by installing pip install dspy-ai, we are unable to view the traces from Langfuse. Should we wait until the next release to utilize that feature? If we install the very latest from main using pip install git+https://github.com/stanfordnlp/dspy.git then also I was not able to see the traces.Could you tell how it needs to be integrated using openai

[xucailiang]

Attached is some relevant documentation on DSPy-Langfuse. you can continue to configure DSPy LMs in the same way, and just do .tracker_call to use Langfuse. lmk if that helps!

@arnavsinghvi11 @xucailiang Could you help in clarifying this? As per the latest DSPy v2.4.13 release, which was done three weeks ago, it doesn't cover the integration of Langfuse with DSPy. If we use the standard DSPy library by installing pip install dspy-ai, we are unable to view the traces from Langfuse. Should we wait until the next release to utilize that feature? If we install the very latest from main using pip install git+https://github.com/stanfordnlp/dspy.git then also I was not able to see the traces.Could you tell how it needs to be integrated using openai

sure, I noticed that you used langfuse's @observe annotation to obtain some metadata, which is one of the recommended methods of langfuse, and you can continue to use this method. The integration of dspy for langfuse appeared after dspy==2.4.13, so if you want to use pip install, you may need to wait for the next official version of dspy to be updated.

In addition.you can also use git clone to pull the latest dspy code and use the tracker module. The following is my test script(the file path is /dspy/test_langfuse.py):

import dspy
turbo = dspy.OpenAI(api_key="your openai key")
# Assuming you have set the environment variables for langfuse
class RAG(dspy.Module):
    def __init__(self):
        super().__init__()
        self.generate_answer = dspy.ChainOfThought("question -> answer")

    def forward(self, question):
        answer = self.generate_answer(question=question)
        print(answer)
        return answer

def test_langfuse():
    prompt = "Hi,how's it going today？(this is a test)"
    result = turbo(prompt)
    print(result)

[mahitha-29]

Attached is some relevant documentation on DSPy-Langfuse. you can continue to configure DSPy LMs in the same way, and just do .tracker_call to use Langfuse. lmk if that helps!

@arnavsinghvi11 @xucailiang Could you help in clarifying this? As per the latest DSPy v2.4.13 release, which was done three weeks ago, it doesn't cover the integration of Langfuse with DSPy. If we use the standard DSPy library by installing pip install dspy-ai, we are unable to view the traces from Langfuse. Should we wait until the next release to utilize that feature? If we install the very latest from main using pip install git+https://github.com/stanfordnlp/dspy.git then also I was not able to see the traces.Could you tell how it needs to be integrated using openai

sure, I noticed that you used langfuse's @observe annotation to obtain some metadata, which is one of the recommended methods of langfuse, and you can continue to use this method. The integration of dspy for langfuse appeared after dspy==2.4.13, so if you want to use pip install, you may need to wait for the next official version of dspy to be updated.

In addition.you can also use git clone to pull the latest dspy code and use the tracker module. The following is my test script(the file path is /dspy/test_langfuse.py):

import dspy
turbo = dspy.OpenAI(api_key="your openai key")
# Assuming you have set the environment variables for langfuse
class RAG(dspy.Module):
    def __init__(self):
        super().__init__()
        self.generate_answer = dspy.ChainOfThought("question -> answer")

    def forward(self, question):
        answer = self.generate_answer(question=question)
        print(answer)
        return answer

def test_langfuse():
    prompt = "Hi,how's it going today？(this is a test)"
    result = turbo(prompt)
    print(result)

thanks for the info!