DataDog / dd-trace-php

Datadog PHP Clients
https://docs.datadoghq.com/tracing/setup/php
Other
481 stars 149 forks source link

[Bug]: Trace ids are no longer cascaded on next spans after `Tracer::extract(Format::TEXT_MAP)` & using context on `startxSpan` #2726

Closed mrtus closed 1 week ago

mrtus commented 1 week ago

Bug report

👋

We use some custom tooling to facilitate better insights through AMQP processes where the resource types are the type of messages, rather than e.g. basic_deliver

After the upgrade the given trace ids are no longer cascaded onto next spans after $spanContext = Tracer::extract(Format::TEXT_MAP, [...]) and use them on Trager::startActiveSpan('operation', ["references" => Reference::create(Reference::CHILD_OF, $spanContext); We already tried startRootSpan as an alternative too. The given span context is created with ::inject(TEXT_MAP)

However the resources are still being registered in the service catalog, but there are no traces at all anymore. We don't seem to understand what changed, and which changes we should make to resolve those issues again.

If we were to publish a message without tracing context to be used, it does create the traces correctly again.

And when debugging a bit more it seems that there is a new trace id created and the given one is being ignored

Implementation details ```php use DDTrace\Contracts\Scope; use DDTrace\Contracts\Tracer; use DDTrace\Exceptions\UnsupportedFormat; use DDTrace\Format; use DDTrace\Reference; use DDTrace\Tag; use Throwable; final class TracingConsumerMiddleware implements ConsumerMiddleware { private const SERVICE_TYPE = 'AMQP'; private const OPERATION_NAME = 'Consumer.listen'; private const PROPERTY_DATADOG_SPAN = 'dd_span'; public function __construct( private readonly Tracer $tracer, private readonly string $serviceName, ) { } public function handle(Message $message, ConsumerHandler $handler): Status { $scope = $this->startActiveSpan($message); $span = $scope->getSpan(); $span->setTag(Tag::SPAN_TYPE, self::SERVICE_TYPE); $span->setTag(Tag::SERVICE_NAME, $this->serviceName); $type = $this->determineTypeFromMessage($message); if ($type !== null) { $span->setResource($type); } try { return $handler->handle($message); } catch (Throwable $exception) { $span->setError($exception); throw $exception; } finally { $scope->close(); $this->tracer->flush(); } } private function determineTypeFromMessage(Message $message): ?string { if ($message->getProperty('type')) { return $message->getProperty('type'); } $payload = json_decode($message->getBody(), true); if (isset($payload['type'])) { return $payload['type']; } return null; } private function startActiveSpan(Message $message): Scope { $propertyDataDogSpan = $message->getProperty(self::PROPERTY_DATADOG_SPAN); if ($propertyDataDogSpan === null) { return $this->tracer->startActiveSpan(self::OPERATION_NAME); } $context = json_decode($propertyDataDogSpan, true, 512, JSON_THROW_ON_ERROR); try { $spanContext = $this->tracer->extract(Format::TEXT_MAP, $context); } catch (UnsupportedFormat) { return $this->tracer->startActiveSpan(self::OPERATION_NAME); } if ($spanContext === null) { return $this->tracer->startActiveSpan(self::OPERATION_NAME); } return $this->tracer->startActiveSpan( self::OPERATION_NAME, [ 'references' => Reference::create(Reference::CHILD_OF, $spanContext), ], ); } } ```
Trace Debug ![image](https://github.com/DataDog/dd-trace-php/assets/8998239/13ba2cc8-46b9-493f-a47f-a8ab711cd2c1)

Would you be able to give us any pointers on what can adjust?

This same code structure did work for us pre-1.x.

Thank you!

PHP version

8.1.x

Tracer or profiler version

1.1.0

Installed extensions

ddtrace is on version 1.1.0

[PHP Modules] apcu bcmath bz2 calendar Core ctype curl datadog-profiling date ddappsec ddtrace dom exif fileinfo filter ftp gd gettext gmp hash iconv igbinary imagick imap intl json libxml mbstring mcrypt memcached mysqli mysqlnd OAuth openssl pcntl pcre PDO pdo_mysql PDO_ODBC pdo_pgsql pdo_sqlite pgsql Phar posix readline redis Reflection session shmop SimpleXML soap sockets sodium SPL sqlite3 standard sysvmsg sysvsem sysvshm tidy tokenizer uuid xml xmlreader xmlwriter xsl Zend OPcache zip zlib

[Zend Modules] Zend OPcache datadog-profiling ddappsec ddtrace

Output of phpinfo()

Our configuration

DD_TRACE_ENABLED=true
DD_TRACE_DEBUG=true
DD_TRACE_AMQP_ENABLED=false
DD_TRACE_CLI_ENABLED=true
DD_AUTOFINISH_SPANS=true
DD_TRACE_GENERATE_ROOT_SPAN=true
DD_TRACE_AUTO_FLUSH_ENABLED=true

Upgrading from

0.99

mrtus commented 1 week ago

Turned out I simply had to use startRootSpan instead of startActiveSpan.