feat(langchain): report llm spans (instead of normal instrumentations)

nirga commented 4 days ago

Fixes #1395, #1297

Instead of relying on the SDKs instrumentations when using LangChain, we should use the fact that frameworks like LangChain and LlamaIndex know about the LLM calls and all the details and already provide them in a standard format. This PR starts reporting LLM calls here and suppresses those reports in the LLM-specific instrumentations.

[ ] I have added tests that cover my changes.
[ ] If adding a new instrumentation or changing an existing one, I've added screenshots from some observability platform showing the change.
[ ] PR name follows conventional commits format: feat(instrumentation): ... or fix(instrumentation): ....
[ ] (If applicable) I have updated the documentation accordingly.

cc: @tibor-reiss

tibor-reiss commented 4 days ago

100%

Something to consider regarding the tests: remove the llm-specific instrumentors when langchain (probably same for llama-index) is used, e.g. https://github.com/traceloop/openllmetry/blob/main/packages/opentelemetry-instrumentation-langchain/tests/conftest.py#L25 Edit: that's what #1453 does, right?

nirga commented 4 days ago

The problem is that when folks use the SDK, it installs these instrumentations anyway, so what you're suggesting will just hide the issue. Parallel to this, I'm working on #1453 which will turn them off in langchain traces. But, this still relays on the context being propagated correctly which doesn't seem to always be the case from your other PR.

traceloop / openllmetry

feat(langchain): report llm spans (instead of normal instrumentations) #1452