Closed jay-steyn closed 3 months ago
Hey @jay-steyn :wave:
At first glance, it appears to me that you aren't using the datadog-agent, especially given the two following pieces of information:
"agent_url": "collector:4317"
[...]
Diagnostics
agent_error => Received HTTP/0.9 when not allowed
Diagnostic checks => failed
As illustrated in this documentation, to leverage the Datadog instrumentation, you'll need to use the Datadog Agent. This way, you'll be able to get both OTel-generated and DDog-generated spans in the Datadog UI.
I believe you should be able to install the Datadog Agent for your docker application using our public documentation (e.g., this one), but I'll be here to help you out if you need any help in the process 😃
For instance, if you are using docker compose and want to trace a service called wordpress
, you could fundamentally have the following:
Your datadog-agent
service:
datadog-agent:
image: datadog/agent:latest
environment:
DD_API_KEY: <your API key>
DD_SITE: <your site, certainly datadoghq.com or datadoghq.eu>
DD_LOGS_ENABLED: true
networks:
- default
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- /proc/:/host/proc/:ro
- /sys/fs/cgroup/:/host/sys/fs/cgroup:ro
#- /opt/datadog-agent/run:/opt/datadog-agent/run:rw
and your traced wordpress
service:
wordpress:
image: whatever
networks:
- default
environment:
<OTel configs, for instance>
DD_AGENT_HOST: datadog-agent
DD_TRACE_OTEL_ENABLED: true
<Other DD configs as needed, for instance>
depends_on:
- datadog-agent
This is just an example, but this should give you the rough picture :)
Hi, okay thanks for this.
I thought you could use Datadog instrumentation WITHOUT the need for the DD Agent and using the OTEL collector exclusively (refer to this https://github.com/DataDog/dd-trace-php/issues/2339)
Hey @jay-steyn - no, it's just about the ability to send spans created by OTEL to datadog.
And the Jaeger traces in the OTEL PR were just to demonstrate that using OTEL spans with an external collector still works when dd-trace-php is enabled, i.e. nothing breaks by the mere presence of dd-trace-php.
Hey @jay-steyn - no, it's just about the ability to send spans created by OTEL to datadog.
And the Jaeger traces in the OTEL PR were just to demonstrate that using OTEL spans with an external collector still works when dd-trace-php is enabled, i.e. nothing breaks by the mere presence of dd-trace-php.
Okay thanks. I will look at another approach.
Hello @bwoebi, I stumbled upon @jay-steyn's issue because I was confused too.
Did I (eventually :) ) understand correctly that Custom Instrumentation of PHP Applications with the OpenTelemetry API (and #2332) is to allow to have both (1) dd-trace-php instrumentation, collected by a Datadog agent, and (2) OpenTelemetry SDK instrumentation, collected by either a Datadog agent or an OTEL collector+Datadog exporter, all in the same application?
This is done by implementing in the dd-trace-php library an OpenTelemetry-compatible tracer, and using that tracer in both instrumentation : the dd-trace-php one, and the OpenTelemetry SDK one. Is that correct?
So that means that an application, instrumented with only dd-trace-php, cannot send traces to an OTEL collector with an otlp
receiver ?
Thank you for your time :)
The purpose of #2332 is to 1. allow the use of other Datadog Products (e.g., Profiling, AppSec) on OTel-generated spans without having to do anything, 2. leverage Datadog's auto-instrumentation, and 3. be able to use the OTel API to instrument your application with open standards (manual or automatic).
have both (1) dd-trace-php instrumentation, collected by a Datadog agent, and (2) OpenTelemetry SDK instrumentation, collected by either a Datadog agent or an OTEL collector+Datadog exporter, all in the same application?
dd-trace-php instrumentation, collected by a Datadog agent,
: Datadog spans created from automatic or manual instrumentation will be collected by the Datadog AgentOpenTelemetry SDK instrumentation, collected by either a Datadog agent or an OTEL collector+Datadog exporter
: OTel-created spans (i.e., Spans created using the OTel API, whether it comes from manual or automatic instrumentation) will use Datadog spans under the hood. The Datadog agent will collect these Datadog spans.This is done by implementing in the dd-trace-php library an OpenTelemetry-compatible tracer
Technically speaking, Datadog's PHP Library is replacing at runtime part of the OTel SDK to ensure interoperability of the APIs—i.e., consistency of information exchange between the Datadog API and the OTel API. Strictly speaking, we are not implementing OTel's Tracer; we are implementing its spans and associated contexts.
and using that tracer in both instrumentation: the dd-trace-php one and the OpenTelemetry SDK one. Is that correct?
Datadog spans are used in both instrumentations, so could you say that the Datadog Tracer is used in both instrumentations... It depends on what you exactly mean by "tracer."
So that means that an application instrumented with only dd-trace-php cannot send traces to an OTEL collector with an OTLP receiver?
That's not the point, indeed. Spans created from the dd-trace-php (i.e., using Datadog's API or from automatic instrumentation) won't be sent to, for instance, an OTel collector. However, spans created using the OTel API should be sent to your OTel collector, as they have always been.
Does that answer your questions? 🙏 Don't hesitate to ask questions. If you're unsure, others may not understand either, so we must improve our documentation and communication. Feedback is very much welcome.
Thank you @PROFeNoM this is much clearer to me, with your detailed explanations.
- allow the use of other Datadog Products (e.g., Profiling, AppSec) on OTel-generated spans without having to do anything, 2. leverage Datadog's auto-instrumentation, and 3. be able to use the OTel API to instrument your application with open standards (manual or automatic).
So, the main target are people that:
I see the need, sadly it's not my need :)
Technically speaking, Datadog's PHP Library is replacing at runtime part of the OTel SDK to ensure interoperability of the APIs—i.e., consistency of information exchange between the Datadog API and the OTel API. Strictly speaking, we are not implementing OTel's Tracer; we are implementing its spans and associated contexts.
I see the big picture, but with the current state of the documentation, I did not understand that.
Spans created from the dd-trace-php (i.e., using Datadog's API or from automatic instrumentation) won't be sent to, for instance, an OTel collector. However, spans created using the OTel API should be sent to your OTel collector, as they have always been.
Do you know if there a plans at Datadog to make an application instrumented with dd-trace-php, able to send signals (or at least traces) to an OTEL collector with an otlp
receiver? Or will Datadog fully embrace the OpenTelemetry PHP API and SDKs and keep dd-trace-php as a RAD, out-of-the-box library, for a smooth onboarding with Datadog?
dd-trace-php have been really stable, well maintained so far, and I don't know if https://github.com/open-telemetry/opentelemetry-php will be as good. They hit their first v1.0 on october which is not that long ago.
_Also, I want to use observability to hunt bug in my code, I don't want to have to hunt bugs in my 3rd party observability code :rolleyes:
Does that answer your questions? 🙏 Don't hesitate to ask questions. If you're unsure, others may not understand either, so we must improve our documentation and communication. Feedback is very much welcome.
Very much so, thank you. It's a real treat to have answer directly from the developers, and that fast.
Regarding 1., also people who want to use OTel API for their code, but also want to use the automatic integrations dd-trace-php provides.
Do you know if there a plans at Datadog to make an application instrumented with dd-trace-php, able to send signals (or at least traces) to an OTEL collector with an otlp receiver? Or will Datadog fully embrace the OpenTelemetry PHP API and SDKs and keep dd-trace-php as a RAD, out-of-the-box library, for a smooth onboarding with Datadog?
Neither is planned, at least currently. Technically, we provide the raw span data with \dd_trace_serialize_closed_spans()
, which could be used at the end of a request to transform our spans into a format otlp understands.
But yes, overall our target are definitely Datadog users. We don't aim to explicitly prevent using our extension for non-Datadog purposes, but we do not officially support it either.
I see the big picture, but with the current state of the documentation, I did not understand that.
Yes, I agree, we're going to look at improving the documentation accordingly, as you're definitely not the first one to misunderstand the capabilities we provide.
Bug report
I have followed the instructions for OTEL tracing. I used out-of-the-box setup for PHP as instructed here https://docs.datadoghq.com/tracing/trace_collection/automatic_instrumentation/dd_libraries/php/
and set
DD_TRACE_OTEL_ENABLED
totrue
I am running PHP 8.0-fpm and Wordpress. I have a Docker run OTEL collector with the Datadog exporters created.
I am not seeing any tracing. If I add OTEL's SDK for tracing, then I see some rudimentary spans. But still nothing from Datadog tracer. I even set the agent to the OTEL collector, as its not clear how you get/set the collector URI.
Do you have a tutorial somewhere where I can see how you got this working clearly? I saw in this PR https://github.com/DataDog/dd-trace-php/pull/2332 very clear view of data in Jaeger and Datadog and would love you know how this was setup correctly.
PHP version
8.0.0
Tracer or profiler version
0.98.1
Installed extensions
[PHP Modules] Core ctype curl date ddappsec ddtrace dom exif fileinfo filter ftp gd grpc hash iconv intl json libxml mbstring mysqli mysqlnd openssl opentelemetry pcre PDO pdo_mysql pdo_sqlite Phar posix protobuf readline Reflection session SimpleXML soap sodium SPL sqlite3 standard tokenizer xml xmlreader xmlwriter xsl Zend OPcache zip zlib
[Zend Modules] Zend OPcache ddappsec ddtrace
Output of
phpinfo()