🗺️ OTEL Instrumentation

[mikeldking]

In order to reach more languages and to play better with the OTEL ecosystem, phoenix will switch over to using OTEL.

Spike

• [x] Investigate llmetry
• [x] #1923

Infra

• [x] Setup monorepo for OpenInference
• [x] https://github.com/Arize-ai/openinference/issues/17
• [x] https://github.com/Arize-ai/openinference/issues/23

  Python

• [x] #1961
• [x] #1957
• [x] #1960
• [x] #2014
• [x] [otel] setup python sub-repo under openinference
• [x] [otel] move instrumentation to openinference repo

Server

• [x] #1952

Javascript

• [x] https://github.com/Arize-ai/openinference/issues/12
• [x] #1962
• [x] #2013

Migration

• [x] #2119
• [x] #2129
• [x] #2130

Documentation

• [x] #2131
• [x] #2132
• [x] #2133
• [x] #2134
• [x] #2135
• [x] #2172
• [x] [openinference] OpenAI Readme
• [x] [openinference] LlamaIndex Readme
• [x] [openinference] LangChain Readme
• [x] [openinference] Bedrock Readme
• [x] [openinference] DSPy Readme
• [x] #2223
• [x] #2224

[dosubot[bot]]

🤖

Hello @mikeldking! Good to see you again. Let's dive into this issue together, shall we?

Based on the information provided, it seems that the Arize Phoenix project is currently using the OpenInference tracing format for capturing and storing LLM Application execution logs. This is evident from the code in the file src/phoenix/trace/llama_index/callback.py where the OpenInferenceTraceCallbackHandler class is defined. This class is a callback handler for storing LLM application trace data in OpenInference format.

However, the project is planning to switch over to using the OTEL ecosystem. The OTEL ecosystem, also known as OpenTelemetry, is a set of APIs, libraries, agents, and instrumentation that can be used for capturing distributed traces and metrics from your application. It's a CNCF project and is designed to be vendor-agnostic.

The migration to the OTEL ecosystem is likely to provide several benefits. For instance, it could potentially provide better support for more languages, better integration with other tools in the OTEL ecosystem, and improved performance. However, without more specific information about the Arize Phoenix project's requirements and goals, it's difficult to provide a more detailed explanation of the potential benefits.

The migration process seems to involve several steps, including setting up a monorepo for OpenInference, moving the instrumentation to the openinference repo, and migrating to OpenInference OpenAI, LlamaIndex, and LangChain Instrumentation.

Unfortunately, I wasn't able to find more specific information about the OTEL ecosystem and how it relates to the Arize Phoenix project within the repository. If you could provide more details about the project's requirements and goals, it would be easier to provide a more accurate and detailed response.

I hope this information is helpful. If you have any further questions or need more clarification, please let me know.

Sources

• src/phoenix/trace/llama_index/callback.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[dosubot[bot]]

🤖

Hello @mikeldking! Good to see you again. Let's dive into this issue together, shall we?

Based on the information provided, it seems that the Arize Phoenix project is currently using the OpenInference tracing format for capturing and storing LLM Application execution logs. This is evident from the code in the file src/phoenix/trace/llama_index/callback.py where the OpenInferenceTraceCallbackHandler class is defined. This class is a callback handler for storing LLM application trace data in OpenInference format.

However, the project is planning to switch over to using the OTEL ecosystem. The OTEL ecosystem, also known as OpenTelemetry, is a set of APIs, libraries, agents, and instrumentation that can be used for capturing distributed traces and metrics from your application. It's a CNCF project and is designed to be vendor-agnostic.

The migration to the OTEL ecosystem is likely to provide several benefits. For instance, it could potentially provide better support for more languages, better integration with other tools in the OTEL ecosystem, and improved performance. However, without more specific information about the Arize Phoenix project's requirements and goals, it's difficult to provide a more detailed explanation of the potential benefits.

The migration process seems to involve several steps, including setting up a monorepo for OpenInference, moving the instrumentation to the openinference repo, and migrating to OpenInference OpenAI, LlamaIndex, and LangChain Instrumentation.

Unfortunately, I wasn't able to find more specific information about the OTEL ecosystem and how it relates to the Arize Phoenix project within the repository. If you could provide more details about the project's requirements and goals, it would be easier to provide a more accurate and detailed response.

I hope this information is helpful. If you have any further questions or need more clarification, please let me know.

Sources

• src/phoenix/trace/llama_index/callback.py

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[amank94]

Docs feedback:

• Setup a server section has typo + ends abruptly: https://docs.arize.com/phoenix/deployment/deploying-phoenix#setup-a-server
• Typo in sagemaker doc ("envoron") : https://docs.arize.com/phoenix/environments
• https://docs.arize.com/phoenix/quickstart/llm-traces --> this page has an out of date OpenAI ChatCompletion endpoint call. See latest here: https://platform.openai.com/docs/api-reference/chat?lang=python

[mikeldking]

python3 -m phoenix.server.main fails even when installed.

[AparnaDhinakaran]

Is there someway to distinguish between local Phoenix server and the hosted server?

I had Phoenix pointing to a hosted environment:

However it still pointed me to the local phoenix server. This didn't have the traces, and makes sense it was in the hosted one.

Traces showed up in hosted:

[fjcasti1]

For spinning up the LLM app + tracing using Phoenix I think the amount of information is great but I would suggest reorganizing. I would first show how to use the deployment via Docker. Once the reader sees the value, I would then add the section of deployment for development where they deploy BE, FE and Phoenix. I was missing how to deploy Phoenix in this case

[fjcasti1]

Outputs of agents do not appear

[fjcasti1]

We should include in the documentation how to extract traces from Phoenix, regardless of how it is deployed

[axiomofjoy]

• Makes examples in openinference repo more prominent.

[mikeldking]

Outputs of agents do not appear

Good catch. This is tracked via https://github.com/Arize-ai/openinference/issues/154

[mikeldking]

Is there someway to distinguish between local Phoenix server and the hosted server?

I had Phoenix pointing to a hosted environment:

However it still pointed me to the local phoenix server. This didn't have the traces, and makes sense it was in the hosted one.

Traces showed up in hosted:

Makes sense that this is confusing. Will file a ticket to make this more clear.