DataDog / dd-trace-php

Datadog PHP Clients
https://docs.datadoghq.com/tracing/setup/php
Other
483 stars 149 forks source link

[Documentation]: OTEL tracing #2566

Closed jay-steyn closed 3 months ago

jay-steyn commented 3 months ago

Bug report

I have followed the instructions for OTEL tracing. I used out-of-the-box setup for PHP as instructed here https://docs.datadoghq.com/tracing/trace_collection/automatic_instrumentation/dd_libraries/php/

and set DD_TRACE_OTEL_ENABLED to true

I am running PHP 8.0-fpm and Wordpress. I have a Docker run OTEL collector with the Datadog exporters created.

I am not seeing any tracing. If I add OTEL's SDK for tracing, then I see some rudimentary spans. But still nothing from Datadog tracer. I even set the agent to the OTEL collector, as its not clear how you get/set the collector URI.

Do you have a tutorial somewhere where I can see how you got this working clearly? I saw in this PR https://github.com/DataDog/dd-trace-php/pull/2332 very clear view of data in Jaeger and Datadog and would love you know how this was setup correctly.

PHP version

8.0.0

Tracer or profiler version

0.98.1

Installed extensions

[PHP Modules] Core ctype curl date ddappsec ddtrace dom exif fileinfo filter ftp gd grpc hash iconv intl json libxml mbstring mysqli mysqlnd openssl opentelemetry pcre PDO pdo_mysql pdo_sqlite Phar posix protobuf readline Reflection session SimpleXML soap sodium SPL sqlite3 standard tokenizer xml xmlreader xmlwriter xsl Zend OPcache zip zlib

[Zend Modules] Zend OPcache ddappsec ddtrace

Output of phpinfo()


Datadog PHP tracer extension
For help, check out the documentation at https://docs.datadoghq.com/tracing/languages/php/
(c) Datadog 2020

Datadog tracing support => enabled
Version => 0.98.1
DATADOG TRACER CONFIGURATION => {
    "date": "2024-03-11T14:30:24Z",
    "os_name": "Linux 31046cfdfcd0 6.6.12-linuxkit #1 SMP Thu Feb  8 06:36:34 UTC 2024 aarch64",
    "os_version": "6.6.12-linuxkit",
    "version": "0.98.1",
    "lang": "php",
    "lang_version": "8.0.30",
    "env": null,
    "enabled": true,
    "service": null,
    "enabled_cli": true,
    "agent_url": "collector:4317",
    "debug": true,
    "analytics_enabled": false,
    "sample_rate": -1,
    "sampling_rules": [],
    "tags": [],
    "service_mapping": [],
    "distributed_tracing_enabled": true,
    "priority_sampling_enabled": true,
    "dd_version": null,
    "architecture": "aarch64",
    "sapi": "cli",
    "datadog.trace.request_init_hook": "\/opt\/datadog\/dd-library\/0.98.1\/dd-trace-sources\/bridge\/dd_wrap_autoloader.php",
    "open_basedir_configured": false,
    "uri_fragment_regex": null,
    "uri_mapping_incoming": null,
    "uri_mapping_outgoing": null,
    "auto_flush_enabled": false,
    "generate_root_span": true,
    "http_client_split_by_domain": true,
    "measure_compile_time": true,
    "report_hostname_on_root_span": false,
    "traced_internal_functions": null,
    "auto_prepend_file_configured": true,
    "integrations_disabled": "default",
    "enabled_from_env": true,
    "opcache.file_cache": null,
    "agent_error": "Received HTTP\/0.9 when not allowed"
}

                               Diagnostics                               
agent_error => Received HTTP/0.9 when not allowed
Diagnostic checks => failed

Directive => Local Value => Master Value
ddtrace.disable => 0 => 0
ddtrace.cgroup_file => /proc/self/cgroup => /proc/self/cgroup
datadog.trace.sidecar_trace_sender => Off => Off
datadog.trace.request_init_hook => /opt/datadog/dd-library/0.98.1/dd-trace-sources/bridge/dd_wrap_autoloader.php => /opt/datadog/dd-library/0.98.1/dd-trace-sources/bridge/dd_wrap_autoloader.php
ddtrace.request_init_hook => /opt/datadog/dd-library/0.98.1/dd-trace-sources/bridge/dd_wrap_autoloader.php => /opt/datadog/dd-library/0.98.1/dd-trace-sources/bridge/dd_wrap_autoloader.php
datadog.trace.agent_url => collector:4317 => collector:4317
datadog.agent_host => no value => no value
datadog.dogstatsd_url => no value => no value
datadog.api_key => no value => no value
datadog.distributed_tracing => On => On
datadog.dogstatsd_port => 8125 => 8125
datadog.env => no value => no value
datadog.autofinish_spans => Off => Off
datadog.trace.url_as_resource_names_enabled => On => On
datadog.http_server_route_based_naming => On => On
datadog.integrations_disabled => default => default
datadog.priority_sampling => On => On
datadog.service => no value => no value
datadog.service_name => no value => no value
datadog.service_mapping => no value => no value
datadog.tags => no value => no value
datadog.trace.global_tags => no value => no value
datadog.trace.agent_port => 0 => 0
datadog.trace.analytics_enabled => Off => Off
datadog.trace.append_trace_ids_to_logs => Off => Off
datadog.trace.auto_flush_enabled => Off => Off
datadog.trace.cli_enabled => On => On
datadog.trace.measure_compile_time => On => On
datadog.trace.debug => On => On
datadog.trace.enabled => On => On
datadog.instrumentation_telemetry_enabled => Off => Off
datadog.trace.health_metrics_enabled => Off => Off
datadog.trace.health_metrics_heartbeat_sample_rate => 0.001 => 0.001
datadog.trace.db_client_split_by_instance => On => On
datadog.trace.http_client_split_by_domain => On => On
datadog.trace.redis_client_split_by_host => Off => Off
datadog.trace.memory_limit => no value => no value
datadog.trace.report_hostname => Off => Off
datadog.trace.flush_collect_cycles => Off => Off
datadog.trace.laravel_queue_distributed_tracing => On => On
datadog.trace.remove_root_span_laravel_queue => On => On
datadog.trace.remove_autoinstrumentation_orphans => Off => Off
datadog.trace.resource_uri_fragment_regex => no value => no value
datadog.trace.resource_uri_mapping_incoming => no value => no value
datadog.trace.resource_uri_mapping_outgoing => no value => no value
datadog.trace.resource_uri_query_param_allowed => no value => no value
datadog.trace.http_url_query_param_allowed => * => *
datadog.trace.http_post_data_param_allowed => no value => no value
datadog.trace.rate_limit => 0 => 0
datadog.trace.sample_rate => -1 => -1
datadog.sampling_rate => -1 => -1
datadog.trace.sampling_rules => [] => []
datadog.trace.sampling_rules_format => regex => regex
datadog.span_sampling_rules => [] => []
datadog.span_sampling_rules_file => no value => no value
datadog.trace.header_tags => no value => no value
datadog.trace.x_datadog_tags_max_length => 512 => 512
datadog.trace.peer_service_mapping => no value => no value
datadog.trace.peer_service_defaults_enabled => Off => Off
datadog.trace.remove_integration_service_names_enabled => Off => Off
datadog.trace.propagate_service => Off => Off
datadog.trace.propagation_style_extract => datadog,tracecontext,B3,B3 single header => datadog,tracecontext,B3,B3 single header
datadog.propagation_style_extract => datadog,tracecontext,B3,B3 single header => datadog,tracecontext,B3,B3 single header
datadog.trace.propagation_style_inject => datadog,tracecontext => datadog,tracecontext
datadog.propagation_style_inject => datadog,tracecontext => datadog,tracecontext
datadog.trace.propagation_style => datadog,tracecontext => datadog,tracecontext
datadog.trace.traced_internal_functions => no value => no value
datadog.trace.agent_timeout => 500 => 500
datadog.trace.agent_connect_timeout => 100 => 100
datadog.trace.debug_prng_seed => -1 => -1
datadog.log_backtrace => On => On
datadog.trace.generate_root_span => On => On
datadog.trace.spans_limit => 1000 => 1000
datadog.trace.128_bit_traceid_generation_enabled => On => On
datadog.trace.128_bit_traceid_logging_enabled => Off => Off
datadog.trace.agent_max_consecutive_failures => 3 => 3
datadog.trace.agent_attempt_retry_time_msec => 5000 => 5000
datadog.trace.bgs_connect_timeout => 2000 => 2000
datadog.trace.bgs_timeout => 5000 => 5000
datadog.trace.agent_flush_interval => 5000 => 5000
datadog.trace.agent_flush_after_n_requests => 10 => 10
datadog.trace.shutdown_timeout => 5000 => 5000
datadog.trace.startup_logs => On => On
datadog.trace.once_logs => On => On
datadog.trace.agent_retries => 0 => 0
datadog.trace.agent_debug_verbose_curl => Off => Off
datadog.trace.debug_curl_output => Off => Off
datadog.trace.beta_high_memory_pressure_percent => 80 => 80
datadog.trace.agentless => Off => Off
datadog.trace.warn_legacy_dd_trace => On => On
datadog.trace.retain_thread_capabilities => Off => Off
datadog.version => no value => no value
datadog.trace.obfuscation_query_string_regexp => (?i)(?:(?:"|%22)?)(?:(?:old[-_]?|new[-_]?)?p(?:ass)?w(?:or)?d(?:1|2)?|pass(?:[-_]?phrase)?|secret|(?:api[-_]?|private[-_]?|public[-_]?|access[-_]?|secret[-_]?|app(?:lication)?[-_]?)key(?:[-_]?id)?|token|consumer[-_]?(?:id|key|secret)|sign(?:ed|ature)?|auth(?:entication|orization)?)(?:(?:\s|%20)*(?:=|%3D)[^&]+|(?:"|%22)(?:\s|%20)*(?::|%3A)(?:\s|%20)*(?:"|%22)(?:%2[^2]|%[^2]|[^"%])+(?:"|%22))|(?:bearer(?:\s|%20)+[a-z0-9._\-]+|token(?::|%3A)[a-z0-9]{13}|gh[opsu]_[0-9a-zA-Z]{36}|ey[I-L](?:[\w=-]|%3D)+\.ey[I-L](?:[\w=-]|%3D)+(?:\.(?:[\w.+/=-]|%3D|%2F|%2B)+)?|-{5}BEGIN(?:[a-z\s]|%20)+PRIVATE(?:\s|%20)KEY-{5}[^\-]+-{5}END(?:[a-z\s]|%20)+PRIVATE(?:\s|%20)KEY(?:-{5})?(?:\n|%0A)?|(?:ssh-(?:rsa|dss)|ecdsa-[a-z0-9]+-[a-z0-9]+)(?:\s|%20|%09)+(?:[a-z0-9/.+]|%2F|%5C|%2B){100,}(?:=|%3D)*(?:(?:\s|%20|%09)+[a-z0-9._-]+)?) => (?i)(?:(?:"|%22)?)(?:(?:old[-_]?|new[-_]?)?p(?:ass)?w(?:or)?d(?:1|2)?|pass(?:[-_]?phrase)?|secret|(?:api[-_]?|private[-_]?|public[-_]?|access[-_]?|secret[-_]?|app(?:lication)?[-_]?)key(?:[-_]?id)?|token|consumer[-_]?(?:id|key|secret)|sign(?:ed|ature)?|auth(?:entication|orization)?)(?:(?:\s|%20)*(?:=|%3D)[^&]+|(?:"|%22)(?:\s|%20)*(?::|%3A)(?:\s|%20)*(?:"|%22)(?:%2[^2]|%[^2]|[^"%])+(?:"|%22))|(?:bearer(?:\s|%20)+[a-z0-9._\-]+|token(?::|%3A)[a-z0-9]{13}|gh[opsu]_[0-9a-zA-Z]{36}|ey[I-L](?:[\w=-]|%3D)+\.ey[I-L](?:[\w=-]|%3D)+(?:\.(?:[\w.+/=-]|%3D|%2F|%2B)+)?|-{5}BEGIN(?:[a-z\s]|%20)+PRIVATE(?:\s|%20)KEY-{5}[^\-]+-{5}END(?:[a-z\s]|%20)+PRIVATE(?:\s|%20)KEY(?:-{5})?(?:\n|%0A)?|(?:ssh-(?:rsa|dss)|ecdsa-[a-z0-9]+-[a-z0-9]+)(?:\s|%20|%09)+(?:[a-z0-9/.+]|%2F|%5C|%2B){100,}(?:=|%3D)*(?:(?:\s|%20|%09)+[a-z0-9._-]+)?)
datadog.trace.client_ip_enabled => Off => Off
datadog.trace.client_ip_header => no value => no value
datadog.trace.forked_process => On => On
datadog.trace.hook_limit => 100 => 100
datadog.trace.agent_max_payload_size => 52428800 => 52428800
datadog.trace.agent_stack_initial_size => 131072 => 131072
datadog.trace.agent_stack_backlog => 12 => 12
datadog.trace.propagate_user_id_default => Off => Off
datadog.dbm_propagation_mode => disabled => disabled
datadog.trace.wordpress_additional_actions => no value => no value
datadog.trace.wordpress_callbacks => On => On
datadog.trace.wordpress_enhanced_integration => On => On
datadog.trace.otel_enabled => On => On
datadog.trace.log_file => no value => no value
datadog.trace.log_level => debug => debug
datadog.trace.amqp_enabled => On => On
datadog.trace.amqp_analytics_enabled => Off => Off
datadog.amqp_analytics_enabled => Off => Off
datadog.trace.amqp_analytics_sample_rate => 1 => 1
datadog.amqp_analytics_sample_rate => 1 => 1
datadog.trace.cakephp_enabled => On => On
datadog.trace.cakephp_analytics_enabled => Off => Off
datadog.cakephp_analytics_enabled => Off => Off
datadog.trace.cakephp_analytics_sample_rate => 1 => 1
datadog.cakephp_analytics_sample_rate => 1 => 1
datadog.trace.codeigniter_enabled => On => On
datadog.trace.codeigniter_analytics_enabled => Off => Off
datadog.codeigniter_analytics_enabled => Off => Off
datadog.trace.codeigniter_analytics_sample_rate => 1 => 1
datadog.codeigniter_analytics_sample_rate => 1 => 1
datadog.trace.exec_enabled => On => On
datadog.trace.exec_analytics_enabled => Off => Off
datadog.exec_analytics_enabled => Off => Off
datadog.trace.exec_analytics_sample_rate => 1 => 1
datadog.exec_analytics_sample_rate => 1 => 1
datadog.trace.curl_enabled => On => On
datadog.trace.curl_analytics_enabled => Off => Off
datadog.curl_analytics_enabled => Off => Off
datadog.trace.curl_analytics_sample_rate => 1 => 1
datadog.curl_analytics_sample_rate => 1 => 1
datadog.trace.drupal_enabled => On => On
datadog.trace.drupal_analytics_enabled => Off => Off
datadog.drupal_analytics_enabled => Off => Off
datadog.trace.drupal_analytics_sample_rate => 1 => 1
datadog.drupal_analytics_sample_rate => 1 => 1
datadog.trace.elasticsearch_enabled => On => On
datadog.trace.elasticsearch_analytics_enabled => Off => Off
datadog.elasticsearch_analytics_enabled => Off => Off
datadog.trace.elasticsearch_analytics_sample_rate => 1 => 1
datadog.elasticsearch_analytics_sample_rate => 1 => 1
datadog.trace.eloquent_enabled => On => On
datadog.trace.eloquent_analytics_enabled => Off => Off
datadog.eloquent_analytics_enabled => Off => Off
datadog.trace.eloquent_analytics_sample_rate => 1 => 1
datadog.eloquent_analytics_sample_rate => 1 => 1
datadog.trace.guzzle_enabled => On => On
datadog.trace.guzzle_analytics_enabled => Off => Off
datadog.guzzle_analytics_enabled => Off => Off
datadog.trace.guzzle_analytics_sample_rate => 1 => 1
datadog.guzzle_analytics_sample_rate => 1 => 1
datadog.trace.laminas_enabled => On => On
datadog.trace.laminas_analytics_enabled => Off => Off
datadog.laminas_analytics_enabled => Off => Off
datadog.trace.laminas_analytics_sample_rate => 1 => 1
datadog.laminas_analytics_sample_rate => 1 => 1
datadog.trace.laravel_enabled => On => On
datadog.trace.laravel_analytics_enabled => Off => Off
datadog.laravel_analytics_enabled => Off => Off
datadog.trace.laravel_analytics_sample_rate => 1 => 1
datadog.laravel_analytics_sample_rate => 1 => 1
datadog.trace.laravelqueue_enabled => On => On
datadog.trace.laravelqueue_analytics_enabled => Off => Off
datadog.laravelqueue_analytics_enabled => Off => Off
datadog.trace.laravelqueue_analytics_sample_rate => 1 => 1
datadog.laravelqueue_analytics_sample_rate => 1 => 1
datadog.trace.logs_enabled => On => On
datadog.logs_injection => On => On
datadog.trace.logs_analytics_enabled => Off => Off
datadog.logs_analytics_enabled => Off => Off
datadog.trace.logs_analytics_sample_rate => 1 => 1
datadog.logs_analytics_sample_rate => 1 => 1
datadog.trace.lumen_enabled => On => On
datadog.trace.lumen_analytics_enabled => Off => Off
datadog.lumen_analytics_enabled => Off => Off
datadog.trace.lumen_analytics_sample_rate => 1 => 1
datadog.lumen_analytics_sample_rate => 1 => 1
datadog.trace.magento_enabled => On => On
datadog.trace.magento_analytics_enabled => Off => Off
datadog.magento_analytics_enabled => Off => Off
datadog.trace.magento_analytics_sample_rate => 1 => 1
datadog.magento_analytics_sample_rate => 1 => 1
datadog.trace.memcache_enabled => On => On
datadog.trace.memcache_analytics_enabled => Off => Off
datadog.memcache_analytics_enabled => Off => Off
datadog.trace.memcache_analytics_sample_rate => 1 => 1
datadog.memcache_analytics_sample_rate => 1 => 1
datadog.trace.memcached_enabled => On => On
datadog.trace.memcached_analytics_enabled => Off => Off
datadog.memcached_analytics_enabled => Off => Off
datadog.trace.memcached_analytics_sample_rate => 1 => 1
datadog.memcached_analytics_sample_rate => 1 => 1
datadog.trace.mongo_enabled => On => On
datadog.trace.mongo_analytics_enabled => Off => Off
datadog.mongo_analytics_enabled => Off => Off
datadog.trace.mongo_analytics_sample_rate => 1 => 1
datadog.mongo_analytics_sample_rate => 1 => 1
datadog.trace.mongodb_enabled => On => On
datadog.trace.mongodb_analytics_enabled => Off => Off
datadog.mongodb_analytics_enabled => Off => Off
datadog.trace.mongodb_analytics_sample_rate => 1 => 1
datadog.mongodb_analytics_sample_rate => 1 => 1
datadog.trace.mysqli_enabled => On => On
datadog.trace.mysqli_analytics_enabled => Off => Off
datadog.mysqli_analytics_enabled => Off => Off
datadog.trace.mysqli_analytics_sample_rate => 1 => 1
datadog.mysqli_analytics_sample_rate => 1 => 1
datadog.trace.nette_enabled => On => On
datadog.trace.nette_analytics_enabled => Off => Off
datadog.nette_analytics_enabled => Off => Off
datadog.trace.nette_analytics_sample_rate => 1 => 1
datadog.nette_analytics_sample_rate => 1 => 1
datadog.trace.pcntl_enabled => On => On
datadog.trace.pcntl_analytics_enabled => Off => Off
datadog.pcntl_analytics_enabled => Off => Off
datadog.trace.pcntl_analytics_sample_rate => 1 => 1
datadog.pcntl_analytics_sample_rate => 1 => 1
datadog.trace.pdo_enabled => On => On
datadog.trace.pdo_analytics_enabled => Off => Off
datadog.pdo_analytics_enabled => Off => Off
datadog.trace.pdo_analytics_sample_rate => 1 => 1
datadog.pdo_analytics_sample_rate => 1 => 1
datadog.trace.phpredis_enabled => On => On
datadog.trace.phpredis_analytics_enabled => Off => Off
datadog.phpredis_analytics_enabled => Off => Off
datadog.trace.phpredis_analytics_sample_rate => 1 => 1
datadog.phpredis_analytics_sample_rate => 1 => 1
datadog.trace.predis_enabled => On => On
datadog.trace.predis_analytics_enabled => Off => Off
datadog.predis_analytics_enabled => Off => Off
datadog.trace.predis_analytics_sample_rate => 1 => 1
datadog.predis_analytics_sample_rate => 1 => 1
datadog.trace.psr18_enabled => On => On
datadog.trace.psr18_analytics_enabled => Off => Off
datadog.psr18_analytics_enabled => Off => Off
datadog.trace.psr18_analytics_sample_rate => 1 => 1
datadog.psr18_analytics_sample_rate => 1 => 1
datadog.trace.roadrunner_enabled => On => On
datadog.trace.roadrunner_analytics_enabled => Off => Off
datadog.roadrunner_analytics_enabled => Off => Off
datadog.trace.roadrunner_analytics_sample_rate => 1 => 1
datadog.roadrunner_analytics_sample_rate => 1 => 1
datadog.trace.sqlsrv_enabled => On => On
datadog.trace.sqlsrv_analytics_enabled => Off => Off
datadog.sqlsrv_analytics_enabled => Off => Off
datadog.trace.sqlsrv_analytics_sample_rate => 1 => 1
datadog.sqlsrv_analytics_sample_rate => 1 => 1
datadog.trace.slim_enabled => On => On
datadog.trace.slim_analytics_enabled => Off => Off
datadog.slim_analytics_enabled => Off => Off
datadog.trace.slim_analytics_sample_rate => 1 => 1
datadog.slim_analytics_sample_rate => 1 => 1
datadog.trace.symfony_enabled => On => On
datadog.trace.symfony_analytics_enabled => Off => Off
datadog.symfony_analytics_enabled => Off => Off
datadog.trace.symfony_analytics_sample_rate => 1 => 1
datadog.symfony_analytics_sample_rate => 1 => 1
datadog.trace.web_enabled => On => On
datadog.trace.web_analytics_enabled => Off => Off
datadog.web_analytics_enabled => Off => Off
datadog.trace.web_analytics_sample_rate => 1 => 1
datadog.web_analytics_sample_rate => 1 => 1
datadog.trace.wordpress_enabled => On => On
datadog.trace.wordpress_analytics_enabled => Off => Off
datadog.wordpress_analytics_enabled => Off => Off
datadog.trace.wordpress_analytics_sample_rate => 1 => 1
datadog.wordpress_analytics_sample_rate => 1 => 1
datadog.trace.yii_enabled => On => On
datadog.trace.yii_analytics_enabled => Off => Off
datadog.yii_analytics_enabled => Off => Off
datadog.trace.yii_analytics_sample_rate => 1 => 1
datadog.yii_analytics_sample_rate => 1 => 1
datadog.trace.zendframework_enabled => On => On
datadog.trace.zendframework_analytics_enabled => Off => Off
datadog.zendframework_analytics_enabled => Off => Off
datadog.trace.zendframework_analytics_sample_rate => 1 => 1
datadog.zendframework_analytics_sample_rate => 1 => 1
[ddtrace] [span] Encoding span 7676179524435903297: trace_id=65ef1580000000006a87449b7a210341, name='Standard input code', service='Standard input code', resource: 'Standard input code', type 'cli' with tags: runtime-id='69ae0c34-c896-4dc1-8e53-fbe865aa1a68', _dd.p.dm='-0', _dd.p.tid='65ef158000000000'; and metrics: process_id='975', _dd.agent_psr='1', _sampling_priority_v1='1', php.compilation.total_time_ms='0'
[ddtrace] [info] Flushing trace of size 1 to send-queue for collector:4317
[ddtrace] [warning] Backing up auto_prepend_file '/var/www/otel/tracing.php'
Extension 'datadog-profiling' not present.
[ddtrace] [span] Encoding span 11442027527812300245: trace_id=65ef1580000000009eca3f6a017e3dd5, name='Standard input code', service='Standard input code', resource: 'Standard input code', type 'cli' with tags: runtime-id='1c408f65-cb30-45ce-99fa-3b6000fd45cf', _dd.p.dm='-0', _dd.p.tid='65ef158000000000'; and metrics: process_id='983', _dd.agent_psr='1', _sampling_priority_v1='1', php.compilation.total_time_ms='0'
[ddtrace] [info] Flushing trace of size 1 to send-queue for collector:4317

### Upgrading from

_No response_
PROFeNoM commented 3 months ago

Hey @jay-steyn :wave:

At first glance, it appears to me that you aren't using the datadog-agent, especially given the two following pieces of information:

"agent_url": "collector:4317"
[...]
                               Diagnostics                               
agent_error => Received HTTP/0.9 when not allowed
Diagnostic checks => failed

As illustrated in this documentation, to leverage the Datadog instrumentation, you'll need to use the Datadog Agent. This way, you'll be able to get both OTel-generated and DDog-generated spans in the Datadog UI.

I believe you should be able to install the Datadog Agent for your docker application using our public documentation (e.g., this one), but I'll be here to help you out if you need any help in the process 😃


For instance, if you are using docker compose and want to trace a service called wordpress, you could fundamentally have the following:

Your datadog-agent service:

  datadog-agent:
    image: datadog/agent:latest
    environment:
      DD_API_KEY: <your API key>
      DD_SITE: <your site, certainly datadoghq.com or datadoghq.eu>
      DD_LOGS_ENABLED: true
    networks:
      - default
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - /proc/:/host/proc/:ro
      - /sys/fs/cgroup/:/host/sys/fs/cgroup:ro
      #- /opt/datadog-agent/run:/opt/datadog-agent/run:rw

and your traced wordpress service:

  wordpress:
    image: whatever
    networks:
      - default
    environment:
      <OTel configs, for instance>
      DD_AGENT_HOST: datadog-agent
      DD_TRACE_OTEL_ENABLED: true
      <Other DD configs as needed, for instance>
    depends_on:
      - datadog-agent

This is just an example, but this should give you the rough picture :)

jay-steyn commented 3 months ago

Hi, okay thanks for this.

I thought you could use Datadog instrumentation WITHOUT the need for the DD Agent and using the OTEL collector exclusively (refer to this https://github.com/DataDog/dd-trace-php/issues/2339)

bwoebi commented 3 months ago

Hey @jay-steyn - no, it's just about the ability to send spans created by OTEL to datadog.

And the Jaeger traces in the OTEL PR were just to demonstrate that using OTEL spans with an external collector still works when dd-trace-php is enabled, i.e. nothing breaks by the mere presence of dd-trace-php.

jay-steyn commented 3 months ago

Hey @jay-steyn - no, it's just about the ability to send spans created by OTEL to datadog.

And the Jaeger traces in the OTEL PR were just to demonstrate that using OTEL spans with an external collector still works when dd-trace-php is enabled, i.e. nothing breaks by the mere presence of dd-trace-php.

Okay thanks. I will look at another approach.

clemherreman commented 3 months ago

Hello @bwoebi, I stumbled upon @jay-steyn's issue because I was confused too.

Did I (eventually :) ) understand correctly that Custom Instrumentation of PHP Applications with the OpenTelemetry API (and #2332) is to allow to have both (1) dd-trace-php instrumentation, collected by a Datadog agent, and (2) OpenTelemetry SDK instrumentation, collected by either a Datadog agent or an OTEL collector+Datadog exporter, all in the same application?

This is done by implementing in the dd-trace-php library an OpenTelemetry-compatible tracer, and using that tracer in both instrumentation : the dd-trace-php one, and the OpenTelemetry SDK one. Is that correct?

So that means that an application, instrumented with only dd-trace-php, cannot send traces to an OTEL collector with an otlp receiver ? 

Thank you for your time :)

PROFeNoM commented 3 months ago

The purpose of #2332 is to 1. allow the use of other Datadog Products (e.g., Profiling, AppSec) on OTel-generated spans without having to do anything, 2. leverage Datadog's auto-instrumentation, and 3. be able to use the OTel API to instrument your application with open standards (manual or automatic).

have both (1) dd-trace-php instrumentation, collected by a Datadog agent, and (2) OpenTelemetry SDK instrumentation, collected by either a Datadog agent or an OTEL collector+Datadog exporter, all in the same application?

  1. dd-trace-php instrumentation, collected by a Datadog agent,: Datadog spans created from automatic or manual instrumentation will be collected by the Datadog Agent
  2. OpenTelemetry SDK instrumentation, collected by either a Datadog agent or an OTEL collector+Datadog exporter: OTel-created spans (i.e., Spans created using the OTel API, whether it comes from manual or automatic instrumentation) will use Datadog spans under the hood. The Datadog agent will collect these Datadog spans.

This is done by implementing in the dd-trace-php library an OpenTelemetry-compatible tracer

Technically speaking, Datadog's PHP Library is replacing at runtime part of the OTel SDK to ensure interoperability of the APIs—i.e., consistency of information exchange between the Datadog API and the OTel API. Strictly speaking, we are not implementing OTel's Tracer; we are implementing its spans and associated contexts.

and using that tracer in both instrumentation: the dd-trace-php one and the OpenTelemetry SDK one. Is that correct?

Datadog spans are used in both instrumentations, so could you say that the Datadog Tracer is used in both instrumentations... It depends on what you exactly mean by "tracer."

So that means that an application instrumented with only dd-trace-php cannot send traces to an OTEL collector with an OTLP receiver?

That's not the point, indeed. Spans created from the dd-trace-php (i.e., using Datadog's API or from automatic instrumentation) won't be sent to, for instance, an OTel collector. However, spans created using the OTel API should be sent to your OTel collector, as they have always been.

Does that answer your questions? 🙏 Don't hesitate to ask questions. If you're unsure, others may not understand either, so we must improve our documentation and communication. Feedback is very much welcome.

clemherreman commented 3 months ago

Thank you @PROFeNoM this is much clearer to me, with your detailed explanations.

  1. allow the use of other Datadog Products (e.g., Profiling, AppSec) on OTel-generated spans without having to do anything, 2. leverage Datadog's auto-instrumentation, and 3. be able to use the OTel API to instrument your application with open standards (manual or automatic).

So, the main target are people that:

I see the need, sadly it's not my need :)

Technically speaking, Datadog's PHP Library is replacing at runtime part of the OTel SDK to ensure interoperability of the APIs—i.e., consistency of information exchange between the Datadog API and the OTel API. Strictly speaking, we are not implementing OTel's Tracer; we are implementing its spans and associated contexts.

I see the big picture, but with the current state of the documentation, I did not understand that.

Spans created from the dd-trace-php (i.e., using Datadog's API or from automatic instrumentation) won't be sent to, for instance, an OTel collector. However, spans created using the OTel API should be sent to your OTel collector, as they have always been.

Do you know if there a plans at Datadog to make an application instrumented with dd-trace-php, able to send signals (or at least traces) to an OTEL collector with an otlp receiver? Or will Datadog fully embrace the OpenTelemetry PHP API and SDKs and keep dd-trace-php as a RAD, out-of-the-box library, for a smooth onboarding with Datadog?

dd-trace-php have been really stable, well maintained so far, and I don't know if https://github.com/open-telemetry/opentelemetry-php will be as good. They hit their first v1.0 on october which is not that long ago.

_Also, I want to use observability to hunt bug in my code, I don't want to have to hunt bugs in my 3rd party observability code :rolleyes:

Does that answer your questions? 🙏 Don't hesitate to ask questions. If you're unsure, others may not understand either, so we must improve our documentation and communication. Feedback is very much welcome.

Very much so, thank you. It's a real treat to have answer directly from the developers, and that fast.

bwoebi commented 3 months ago

Regarding 1., also people who want to use OTel API for their code, but also want to use the automatic integrations dd-trace-php provides.

Do you know if there a plans at Datadog to make an application instrumented with dd-trace-php, able to send signals (or at least traces) to an OTEL collector with an otlp receiver? Or will Datadog fully embrace the OpenTelemetry PHP API and SDKs and keep dd-trace-php as a RAD, out-of-the-box library, for a smooth onboarding with Datadog?

Neither is planned, at least currently. Technically, we provide the raw span data with \dd_trace_serialize_closed_spans(), which could be used at the end of a request to transform our spans into a format otlp understands.

But yes, overall our target are definitely Datadog users. We don't aim to explicitly prevent using our extension for non-Datadog purposes, but we do not officially support it either.

I see the big picture, but with the current state of the documentation, I did not understand that.

Yes, I agree, we're going to look at improving the documentation accordingly, as you're definitely not the first one to misunderstand the capabilities we provide.