All transactions unknown & 100% error rate post PHP8.1 upgrade (laravel v8)

[TheFrankman]

Update on this issue

The error rate being 100% was actually legitimate and I have resolved it BUT all transactions are still being labelled as unknown.

Description

1 day after switching to the new release all of the transactions are labeled as unknown and the error rate is 100% despite no errors being thrown.

I have opened a forum discussion for this but I have had no response : https://discuss.newrelic.com/t/php-new-relic-completely-broken-after-php-8-1-upgrade/179337

Relevant Logs / Console output

DAEMON STATUS

sudo service newrelic-daemon status
* newrelic-daemon.service - LSB: The New Relic Proxy Daemon
   Loaded: loaded (/etc/init.d/newrelic-daemon; generated)
   Active: active (exited) since Thu 2022-03-10 09:54:58 UTC; 1 weeks 0 days ago
     Docs: man:systemd-sysv-generator(8)
  Process: 17762 ExecStart=/etc/init.d/newrelic-daemon start (code=exited, status=0/SUCCESS)

Mar 10 09:54:58 [hidden] systemd[1]: Starting LSB: The New Relic Proxy Daemon...
Mar 10 09:54:58[hidden] newrelic-daemon[17762]: New Relic Daemon: newrelic-daemon already running
Mar 10 09:54:58 [hidden] systemd[1]: Started LSB: The New Relic Proxy Daemon.

PHP AGENT LOGS

2022-03-14 09:38:20.292 +0000 (18259 18259) warning: daemon connect(fd=16 uds=@newrelic) returned -1 errno=ECONNREFUSED. Failed to connect to the newrelic-daemon. Please make sure that there is a properly configured newrelic-daemon running. For additional assistance, please see: [Starting the PHP daemon (advanced) | New Relic Documentation](https://docs.newrelic.com/docs/apm/agents/php-agent/advanced-installation/starting-php-daemon-advanced/)
2022-03-14 09:39:01.420 +0000 (19119 19119) info: attempt daemon connection via ‘[@newrelic](https://discuss.newrelic.com/u/newrelic)’
2022-03-14 09:39:01.420 +0000 (19119 19119) info: New Relic 9.19.0.309 (“zomp” - “8536ad58490a”) [daemon=’[@newrelic](https://discuss.newrelic.com/u/newrelic)’ php=‘8.1.2’ zts=no sapi=‘cli’ pid=19119 ppid=19108 uid=0 euid=0 gid=0 egid=0 backtrace=yes startup=init os=‘Linux’ rel=‘4.19.0-17-amd64’ mach=‘x86_64’ ver=’#1 SMP Debian 4.19.194-2 (2021-06-21)’ node=‘hidden’]
2022-03-14 09:39:01.488 +0000 (19132 19132) info: attempt daemon connection via ‘[@newrelic](https://discuss.newrelic.com/u/newrelic)’
2022-03-14 09:39:01.488 +0000 (19132 19132) info: New Relic 9.19.0.309 (“zomp” - “8536ad58490a”) [daemon=’[@newrelic](https://discuss.newrelic.com/u/newrelic)’ php=‘8.1.2’ zts=no sapi=‘cli’ pid=19132 ppid=19108 uid=0 euid=0 gid=0 egid=0 backtrace=yes startup=init os=‘Linux’ rel=‘4.19.0-17-amd64’ mach=‘x86_64’ ver=’#1 SMP Debian 4.19.194-2 (2021-06-21)’ node=‘hidden’]
2022-03-14 09:39:01.550 +0000 (19145 19145) info: attempt daemon connection via ‘[@newrelic](https://discuss.newrelic.com/u/newrelic)’
2022-03-14 09:39:01.550 +0000 (19145 19145) info: New Relic 9.19.0.309 (“zomp” - “8536ad58490a”) [daemon=’[@newrelic](https://discuss.newrelic.com/u/newrelic)’ php=‘8.1.2’ zts=no sapi=‘cli’ pid=19145 ppid=19108 uid=0 euid=0 gid=0 egid=0 backtrace=yes startup=init os=‘Linux’ rel=‘4.19.0-17-amd64’ mach=‘x86_64’ ver=’#1 SMP Debian 4.19.194-2 (2021-06-21)’ node=‘hidden’]
2022-03-14 09:52:28.664 +0000 (19209 19209) error: TXNDATA failure: len=8180 errno=EPIPE
2022-03-14 09:52:30.232 +0000 (17106 17106) error: APPINFO failure: len=6340 errno=EPIPE
2022-03-14 09:52:30.997 +0000 (19195 19195) error: TXNDATA failure: len=12332 errno=EPIPE
2022-03-14 09:52:31.026 +0000 (19193 19193) error: APPINFO failure: len=6340 errno=EPIPE
2022-03-14 09:52:31.380 +0000 (19216 19216) error: APPINFO failure: len=6340 errno=EPIPE
2022-03-14 09:52:32.692 +0000 (18355 18355) error: TXNDATA failure: len=11732 errno=EPIPE
2022-03-14 09:52:34.194 +0000 (19214 19214) error: APPINFO failure: len=6340 errno=EPIPE
2022-03-14 09:52:37.828 +0000 (19217 19217) error: APPINFO failure: len=6340 errno=EPIPE
2022-03-14 09:52:38.016 +0000 (19194 19194) error: APPINFO failure: len=6340 errno=EPIPE
2022-03-14 09:52:39.934 +0000 (17863 17863) error: TXNDATA failure: len=12332 errno=EPIPE

Daemon logs from today are empty

Your Environment

Debian GNU/Linux 10 (buster)
PHP 8.1.2
Laravel Framework ^8.34

We are running out of the box newrelic.ini settings and config template

Additional context

[trothwell-carsguide]

@chris-skilltech while the package does help it doesn't seem to fix error reporting

[adduc]

@trothwell-carsguide We've been using the following in our app\Exceptions\Handler.php to handle error reporting:

public function register()
{
    $this->reportable(function (Throwable $e) {
        if (function_exists('newrelic_notice_error')) {
            newrelic_notice_error($e);
        }
    });
}

[carltondickson]

Using the middleware from above that contains newrelic_name_transaction worked but our transaction are coming through as

WebTransaction/Custom/OurAccountName\SearchController@search instead of WebTransaction/Action/OurAccountName\SearchController@search

So just be aware of that.

Upgrading to New Relic 10 alone didn't fix this for us FYI

[dbaggott]

@ak-war, re:

we are expecting a release sometime in October 2022 that will address this issue.

Any updates on when in October the release will be?

[quickbutik-ronflorax]

Come on now @ak-war we pay a lot of money for this service which has been useless for months. No updates at all from your end. We deserve a reply that's better than "We don't know, we hope soon".

[sonus21]

Unfortunately, this is even not working on the dev branch. I tried to compile the PHP agent from the source still it fails.

RUN git clone https://github.com/newrelic/newrelic-php-agent.git && cd newrelic-php-agent && git checkout dev &&  \
    make all && make agent-install && cp bin/daemon /usr/local/bin/nr-daemon && make clean && cd .. && rm -rf  newrelic-php-agent

I'm testing newrelic integration with Laravel, drop is the point when I restarted my docker.

[brensneves]

If this service doesn't work, then why do we pay for it?

[brusin]

If this service doesn't work, then why do we pay for it?

Tbh, we are planning on moving to Sentry in the next few days (our release_candidate is already using it)

[JohnD87]

Any update here? We will be forced to switch to another product in a few weeks if we don't get a solution.

[ak-war]

Thank you everyone for your constructive feedback. A release will be made around November 3 that will be fixing this issue.

[EvanRijn]

@ak-war Is that a final release or a dev release ? Can we test this ?

[ak-war]

@EvanRijn It will be an official release, we do not have a dev release.

[EvanRijn]

@ak-war perfect! is tomorrow still the release day ?

[ak-war]

New Relic PHP Agent v10.3.0.315 has been released with the fix: https://github.com/newrelic/newrelic-php-agent/releases.

[nsinisterra]

Docs referenced on the release notes are not working:

• PHP: Configure logs in context
• PHP agent configuration: Config file.
• Disable automatic logging

[mfulb]

@nsinisterra thanks for catching that - links should be correct now.

[nsinisterra]

Awesome team! I'm able to confirm that this update fix the problem! No other issues for now.

NOTE: For other doing the update right now, we have 4 apps running the APM Agent. In 3 of them, make an apt upgrade was enough, but one of the servers have requested the license key again to update, get your licence key before start the process. Regards!

[harleymckenzie]

Unfortunately we're still having the same issue on the new agent (10.3.0.315-1). Seeing a lot of messages in the php_agent logs with:

2022-11-04 16:44:10.946 +0000 (10460 10460) warning: User instrumentation from opcache: missing 'scripts' key in status information

[theloveofcode]

I have confirmed that we've successfully updated the agent using yum info newrelic-php5 and newrelic-daemon -v. However, running Magento 2.4.4-p2, which listed as explicitly compatible with the PHP agent, we are still seeing everything clumped together under index.php in APM Transctions.

Are there any other steps we need to take other than installing the new agent, or is this issue not really fixed?

[EvanRijn]

nsinisterra

Can you share the installed module list ?

theloveofcode

Can you share the installed module list ?

[theloveofcode]

I'm a bit unclear. The supported operating systems lists CentOS 6 or higher, but then states CentOS 6 support was dropped. Is CentOS 7 supported?

[harleymckenzie]

However, running Magento 2.4.4-p2, which listed as explicitly compatible

Our customers are also using Magento 2.4.4 (and up) are also affected, and also the only ones using PHP 8.1. Those using versions under 2.4.4 for Magento and < PHP 8.1 are having no issues.

[tdgroot]

We're also receiving issues about all requests marked as index.php although we're on the latest 10.3.0.315 version. This is with Magento 2.4.x (I believe 2.4.4 and up) and PHP 8.1.

What strikes to me is that prior to the 10.3.0.315 version, we were having issues that transactions were named as either index.php or unknown. After updating to the 10.3.0.315 version, both problems seemed to disappear, but since yesterday we have seen the index.php grouped transactions reappearing again:

Please let us know/keep us posted about the status of this problem, as many merchants and developers are preparing their applications to be ready for both black friday and the holiday season in general.

[bduranleau-nr]

@theloveofcode or @tdgroot - Would one of you mind opening a new github issue on the repo so we can track the Magento 2 impact? We'd like to increase visibility on this issue and separate it from this Laravel-specific issue. We believe this and #436 were resolved by the 10.3.0 release.

[bduranleau-nr]

Resolved by 10.3.0 release. Please re-open if the issue re-occurs.

[TheFrankman]

I upgraded very pessimistically but i am happy to report that at least for the last hour, transactions are now correct labelled and not unknown.

Let's hope that support for PHP8.2 takes less than a year once it's released.

(Laravel V9, New Relic 10.3.0, PHP8.1)

[jonathanribas]

Hi @tdgroot, hope you doing well.

Are you still experiencing an issue with NR 10.3.0.315 version on Magento 2.4.x (I believe 2.4.4 and up) and PHP 8.1?

Thanks in advance for your help!