The Langfuse integration currently counts the embedding response (the vector data) as output tokens of the observation. These vectors are quite large, often around 30,000 tokens for an OpenAI text-embedding-3-large embedding vector. This overshadows the actual generated output tokens (from completion models) of the entire trace, making it difficult to see how many tokens were genuinely generated, especially in the trace list view.
I do not believe displaying the entire embedding vector as the output provides any value in the Langfuse UI. At the very least, the output vector should not count towards the total used tokens for this particular observation and the whole trace.
What happened?
The Langfuse integration currently counts the embedding response (the vector data) as output tokens of the observation. These vectors are quite large, often around 30,000 tokens for an OpenAI
text-embedding-3-largeembedding vector. This overshadows the actual generated output tokens (from completion models) of the entire trace, making it difficult to see how many tokens were genuinely generated, especially in the trace list view.I do not believe displaying the entire embedding vector as the output provides any value in the Langfuse UI. At the very least, the output vector should not count towards the total used tokens for this particular observation and the whole trace.
Here is a screenshots that shows this problem.
Relevant log output
No response
Twitter / LinkedIn details
No response