search

DataDog / dd-trace-php

Datadog PHP Clients
https://docs.datadoghq.com/tracing/setup/php
Other
496 stars 155 forks source link

[Bug]: [Heroku] [Stack: heroku-24] Previously working deployments suddenly stopped with "ERROR: Required PHP extension 'json' not found." #2787

Closed martynling closed 2 months ago

martynling commented 2 months ago

Bug report

We've been running the heroku-24 stack with PHP 8.3 and able to deploy no problem for over a month. Starting around Aug 7th or 8th, deployments stopped working due to dd-trace issue below:

-----> Downloading Datadog Agent 7.55.1-1
       Reading package lists...
       Building dependency tree...
       Recommended packages:
         datadog-signing-keys
       The following NEW packages will be installed:
         datadog-heroku-agent
       0 upgraded, 1 newly installed, 0 to remove and 26 not upgraded.
       Need to get 114 MB of archives.
       After this operation, 495 MB of additional disk space will be used.
       Get:1 https://apt.datadoghq.com/ stable/7 amd64 datadog-heroku-agent amd64 1:7.55.1-1 [114 MB]
       Fetched 114 MB in 1s (200 MB/s)
       Download complete and in download only mode
W: --force-yes is deprecated, use one of the options starting with --allow instead.
-----> Installing Datadog Agent
-----> DD_PROCESS_AGENT not set. Removing the process agent.
-----> Installing Datadog runner
-----> Installing Datadog agent wrapper
-----> Datadog PHP extensions app detected
-----> Preparing to install Datadog PHP extensions
       Using 'latest' dd-trace-php release (default).
-----> Downloading installer from https://github.com/DataDog/dd-trace-php/releases/latest/download/datadog-setup.php
-----> Running Datadog installer
       Profiling will be enabled
       Appsec will be enabled
       Searching for available php binaries, this operation might take a while.
       Downloading installable archive from https://github.com/DataDog/dd-trace-php/releases/download/1.2.0/dd-library-php-1.2.0-x86_64-linux-gnu.tar.gz.
       This operation might take a while.
       ...................
       Download completed

       Installed required source files to '/tmp/build_05c969f4/opt/dd-library/1.2.0'
       Installing to binary: php (/tmp/build_05c969f4/.heroku/php/bin/php)
       Installing extension to /app/.heroku/php/lib/php/extensions/no-debug-non-zts-20230831
       Copied '/tmp/dd-install/dd-library-php/trace/ext/20230831/ddtrace.so' to '/app/.heroku/php/lib/php/extensions/no-debug-non-zts-20230831/ddtrace.so'
       Copied '/tmp/dd-install/dd-library-php/profiling/ext/20230831/datadog-profiling.so' to '/app/.heroku/php/lib/php/extensions/no-debug-non-zts-20230831/datadog-profiling.so'
       Copied '/tmp/dd-install/dd-library-php/appsec/ext/20230831/ddappsec.so' to '/app/.heroku/php/lib/php/extensions/no-debug-non-zts-20230831/ddappsec.so'
       Created INI file '/app/.heroku/php/etc/php/conf.d/98-ddtrace.ini'
       Installation to 'php (/tmp/build_05c969f4/.heroku/php/bin/php)' was successful
       Installing to binary: php-fpm (/tmp/build_05c969f4/.heroku/php/sbin/php-fpm)
       Killed
       PHP Deprecated:  explode(): Passing null to parameter #2 ($string) of type string is deprecated in /tmp/codon/tmp/buildpacks/766a320daf1a9dc2c1b3c389285af80d17945984/datadog-setup.php on line 1106
       ERROR: Required PHP extension 'json' not found.

 !     Push rejected, failed to compile Datadog PHP extensions app.
 !     Push failed

PHP version, profiler version and other outputs are from the latest running production dyno. I haven't yet been able to find a way to get that info from the dyno that is being built and is generating the above output including the error.

From the error message, it would seem that line 1104 below is returning null unexpectedly:

$extensions = shell_exec("$binary -m");

I'm also reporting this to Heroku as I don't see any indication that there has been a code change related to this in the buildpack, but logging it here in case others are also encountering the same issue.

PHP version

8.3.10

Tracer or profiler version

1.2.0

Installed extensions

php -m [PHP Modules] bz2 Core ctype curl datadog-profiling date ddappsec ddtrace dom fileinfo filter hash iconv json libxml mbstring mysqli mysqlnd openssl pcre PDO pdo_mysql pdo_pgsql pgsql Phar posix random readline Reflection session SimpleXML sockets SPL standard tokenizer xml xmlreader xmlwriter Zend OPcache zip zlib

[Zend Modules] Zend OPcache datadog-profiling ddappsec ddtrace

Output of phpinfo()

No response

Upgrading from

No response

iamluc commented 2 months ago

Hi @martynling Are you still encountering the same issue?

iamluc commented 2 months ago

Also, are you using this buildpack: https://github.com/SpeedCurve-Metrics/heroku-buildpack-php-ddtrace ?

martynling commented 2 months ago

Yes, we still have the same error if we try to deploy to the heroku-24 stack and we are using that buildpack. We have been able to temporarily unblock by reverting to the heroku-22 stack, but on another app (thankfully just on dev) we have also encountered exactly the same error with the heroku-22 stack, so I'm not confident that's going to keep working.

martynling commented 2 months ago

FYI Heroku support doesn't cover 3rd party buildpacks so they were not interested in helping.

realFlowControl commented 2 months ago

Hey @martynling,

thanks for your patience. I am not 100% sure what happens and why the php-fpm -m process gets killed. But as Heroku only supports PHP >=8, json comes bundled anyway, so we decided to soften the check and not fail in case the command failed (see #2805).

I just tried with a small instance and it seems to work:

remote:        
remote:        Installed required source files to '/tmp/build_476efa66/opt/dd-library/1.3.0+9747b3855b2214f785e8a320fb23e68cb0712a7a'
remote:        Installing to binary: php (/tmp/build_476efa66/.heroku/php/bin/php)
remote:        Installing extension to /app/.heroku/php/lib/php/extensions/no-debug-non-zts-20230831
remote:        Copied '/tmp/dd-install/dd-library-php/trace/ext/20230831/ddtrace.so' to '/app/.heroku/php/lib/php/extensions/no-debug-non-zts-20230831/ddtrace.so'
remote:        Copied '/tmp/dd-install/dd-library-php/profiling/ext/20230831/datadog-profiling.so' to '/app/.heroku/php/lib/php/extensions/no-debug-non-zts-20230831/datadog-profiling.so'
remote:        Copied '/tmp/dd-install/dd-library-php/appsec/ext/20230831/ddappsec.so' to '/app/.heroku/php/lib/php/extensions/no-debug-non-zts-20230831/ddappsec.so'
remote:        Created INI file '/app/.heroku/php/etc/php/conf.d/98-ddtrace.ini'
remote:        Installation to 'php (/tmp/build_476efa66/.heroku/php/bin/php)' was successful
remote:        Installing to binary: php-fpm (/tmp/build_476efa66/.heroku/php/sbin/php-fpm)
remote:        Killed
remote:        WARNING: The output of '/tmp/build_476efa66/.heroku/php/sbin/php-fpm -m' could not be reliably checked. Please make sure you have the PHP extension 'json' installed.
remote:        Installing extension to /app/.heroku/php/lib/php/extensions/no-debug-non-zts-20230831
remote:        Copied '/tmp/dd-install/dd-library-php/trace/ext/20230831/ddtrace.so' to '/app/.heroku/php/lib/php/extensions/no-debug-non-zts-20230831/ddtrace.so'
remote:        Copied '/tmp/dd-install/dd-library-php/profiling/ext/20230831/datadog-profiling.so' to '/app/.heroku/php/lib/php/extensions/no-debug-non-zts-20230831/datadog-profiling.so'
remote:        Copied '/tmp/dd-install/dd-library-php/appsec/ext/20230831/ddappsec.so' to '/app/.heroku/php/lib/php/extensions/no-debug-non-zts-20230831/ddappsec.so'
remote:        Updating existing INI file '/app/.heroku/php/etc/php/conf.d/98-ddtrace.ini'
remote:        Installation to 'php-fpm (/tmp/build_476efa66/.heroku/php/sbin/php-fpm)' was successful
remote:        --------------------------------------------------
remote:        SUCCESS
remote:        
remote: -----> Validating installation
remote:        PHP 8.3.10 (cli) (built: Aug  2 2024 12:21:49) (NTS)
remote:        Copyright (c) The PHP Group
remote:        Zend Engine v4.3.10, Copyright (c) Zend Technologies
remote:            with Zend OPcache v8.3.10, Copyright (c), by Zend Technologies
remote:            with ddtrace v1.3.0+9747b3855b2214f785e8a320fb23e68cb0712a7a, Copyright Datadog, by Datadog
remote:            with datadog-profiling v1.3.0+9747b3855b2214f785e8a320fb23e68cb0712a7a, Copyright Datadog, by Datadog
remote:            with ddappsec v1.3.0+9747b3855b2214f785e8a320fb23e68cb0712a7a, Copyright Datadog, by Datadog

This will make it into the next release, I used the datadog-setup.php from that PR to try it out.

You could try using that datadog-setup.php via the DD_INSTALLER_URL config var from the buildpack and see if this fixes the problem for you too.

Kind regards Florian

martynling commented 2 months ago

@realFlowControl That's great news! Thanks for looking into that and finding a solution. And thanks @iamluc too.

bwoebi commented 2 months ago

That "next release" has been released now as 1.3.0. So it should work with the proper release too now.