[Bug]: Broken times in Langfuse?

[Manouchehri]

What happened?

Not sure why, but it seems like timing in Langfuse (from LiteLLM) got borked after I updated to v1.34.25.

Relevant log output

No response

Twitter / LinkedIn details

https://www.linkedin.com/in/davidmanouchehri/

[krrishdholakia]

don't think we changed anything there.

I'll look into this though

[Manouchehri]

When I looked at the data Langfuse is returning to the browser, I noticed it only has startTime and endTime from LiteLLM. Maybe we could send more about the timing info? https://langfuse.com/docs/sdk/typescript/guide

https://github.com/BerriAI/litellm/blob/90e03399fa594f8ac0ed907671feefc402c0b163/litellm/integrations/langfuse.py#L206-L221

https://github.com/BerriAI/litellm/blob/90e03399fa594f8ac0ed907671feefc402c0b163/litellm/integrations/langfuse.py#L327-L339

[
    {
        "result": {
            "data": {
                "json": {
                    "id": "12725486-b0b1-4d8d-b1c3-54d914c4cbb9",
                    "externalId": null,
                    "timestamp": "2024-04-06T12:19:53.129Z",
                    "name": "litellm-acompletion",
                    "userId": "removed",
                    "metadata": null,
                    "release": null,
                    "version": null,
                    "projectId": "removed",
                    "public": false,
                    "bookmarked": false,
                    "tags": [
                        "user_api_key:removed",
                        "user_api_key_user_id:removed",
                        "user_api_key_team_id:removed",
                        "cache_hit:False"
                    ],
                    "input": {
                        "messages": [
                            {
                                "role": "user",
                                "content": "what do you know about V8?"
                            }
                        ]
                    },
                    "output": {
                        "role": "assistant",
                        "content": "removed"
                    },
                    "sessionId": null,
                    "scores": [],
                    "latency": null,
                    "observations": [
                        {
                            "id": "time-12-19-49-345511_chatcmpl-7f024fb0-e144-426d-89e5-ad945e416921",
                            "traceId": "12725486-b0b1-4d8d-b1c3-54d914c4cbb9",
                            "projectId": "removed",
                            "type": "GENERATION",
                            "startTime": "2024-04-06T12:19:53.129Z",
                            "endTime": null,
                            "name": "litellm-acompletion",
                            "metadata": {
                                "team_id": "removed",
                                "deployment": "bedrock/anthropic.claude-3-haiku-20240307-v1:0",
                                "model_info": {
                                    "id": "removed"
                                },
                                "model_group": "claude-3-haiku-20240307",
                                "user_api_key": "removed",
                                "user_api_key_alias": "removed",
                                "user_api_key_team_id": "removed",
                                "user_api_key_user_id": "removed",
                                "user_api_key_metadata": {}
                            },
                            "parentObservationId": null,
                            "level": "DEFAULT",
                            "statusMessage": null,
                            "version": null,
                            "createdAt": "2024-04-06T12:19:54.404Z",
                            "model": "anthropic.claude-3-haiku-20240307-v1:0",
                            "modelParameters": null,
                            "promptTokens": 15,
                            "completionTokens": 291,
                            "totalTokens": 306,
                            "unit": "TOKENS",
                            "completionStartTime": "2024-04-06T12:19:49.986Z",
                            "promptId": null,
                            "modelId": null,
                            "inputPrice": null,
                            "outputPrice": null,
                            "totalPrice": null,
                            "calculatedInputCost": null,
                            "calculatedOutputCost": null,
                            "calculatedTotalCost": "0.0003675"
                        }
                    ]
                },
                "meta": {
                    "values": {
                        "timestamp": [
                            "Date"
                        ],
                        "latency": [
                            "undefined"
                        ],
                        "observations.0.startTime": [
                            "Date"
                        ],
                        "observations.0.createdAt": [
                            "Date"
                        ],
                        "observations.0.completionStartTime": [
                            "Date"
                        ],
                        "observations.0.calculatedTotalCost": [
                            [
                                "custom",
                                "decimal.js"
                            ]
                        ]
                    }
                }
            }
        }
    }
]

[Manouchehri]

I'm working on a bit of this atm. :)

[krrishdholakia]

Thanks! @Manouchehri