langfuse / langfuse

🪢 Open source LLM engineering platform: Observability, metrics, evals, prompt management, playground, datasets. Integrates with LlamaIndex, Langchain, OpenAI SDK, LiteLLM, and more. 🍊YC W23
https://langfuse.com/docs
Other
4.64k stars 426 forks source link

feat: add non-numeric scores via sdks #2514

Open xinyiqin opened 3 days ago

xinyiqin commented 3 days ago

Describe the bug

why scores can be set as categorical type by user annotations, but cannot be set in python SDK? all scores are NUMERIC type now.

To reproduce

langfuse.score( id=id, name="intent", trace_id=generation.trace_id, observation_id=generation.id, data_type='CATEGORICAL', value=0, string_value=str(intent_lst), comment=str(intent_lst), )

SDK and container versions

No response

Additional information

No response

Are you interested to contribute a fix for this bug?

No

marcklingen commented 3 days ago

Thanks for reporting. Yes, this is a current limitation. We will release new APIs and SDK versions to support non-numeric score ingestion. ETA: this week