search

docker-library / php

Docker Official Image packaging for PHP
https://php.net
MIT License
3.77k stars 2k forks source link

30-40% CPU usage increase with 8.2.8-bookworm #1431

Open tlamy opened 10 months ago

tlamy commented 10 months ago

We recently updated from 8.2.7-fpm to 8.2.8-fpm, where base system changed from bullseye to bookworm. After a bit fiddling to re-enable blowfish encryption in OpenSSL 3, I noticed heavily increased cpu usage in all our deployments. After basing our images on 8.2-fpm-bullseye (plus build and deploy), CPU usage is down to normal again.

What could have caused that behavior? How long do you plan to provide bullseye based images? What can I do to help pin down the problem?

tlamy commented 10 months ago

image We deployed on 11:55.

Neither Dockerfiles nor code have changed significantly, and CPU usage was comparable with as we used 8.2.7 (bullseye)

yosifkit commented 10 months ago

How long do you plan to provide bullseye based images?

We'll have bullseye based images until the next Debian stable major release (trixie). It doesn't have a release date but, judging by past releases, it'll likely be around mid 2025 (https://wiki.debian.org/DebianReleases).

What could have caused that behavior?

Not sure. We've had issues in the past that our images are slower than the equivalent Debian packages but have been unable to pinpoint exactly why (https://github.com/docker-library/php/issues/493).

What can I do to help pin down the problem?

You could test to see if the Debian PHP 8.2 packages in Bookworm exhibit the same amount of CPU usage. If they do have similar usage, then maybe a package update in Bookworm is the problem, otherwise it might be something in the way we compile PHP 😢.

tlamy commented 10 months ago

Just a small heads-up, I'm still at it trying to pinpoint the original cause. I managed to create an image using Ondrey Sury's PHP packages for Debian, but still struggling to create some test code, since I don't want to test in production.

magnetik commented 8 months ago

I face the same problem. Not sure on how to pinpoint the origin.

pereorga commented 6 months ago

@magnetik @tlamy can you produce a simple example to try to reproduce the issue?

particleflux commented 5 months ago

Similar issue, but not restricted to this docker image. We run an image based on debian:bookworm-20230919-slim directly. In general, switched from 8.0 to 8.2 (and upgraded from debian buster to bookworm base image).

So I guess it's either bookworm related, or php-fpm 8.2 related, or the packaging by deb.sury.org (which disabled JIT between those: https://github.com/oerdnj/deb.sury.org/issues/1924)

2024-01-16_164010

Unfortunately, can't easily reproduce it either, it only happens on our prod env, with like 300 requests/min

GreenReaper commented 3 months ago

Seems like the thing to do is try re-enabling the tracing JIT and giving it a suitable buffer depending on the codebase, assuming this can be done without too much trouble.

yuqi commented 1 month ago

I have encountered the same issue. The same problem occurred after we upgraded from php:7.4-fpm-bullseye to php:8.1-fpm (bookworm), where P95 and AVG response time increased. Finally, it returned to normal after using php:8.1-fpm-bullseye.

tianon commented 1 month ago

Having a minimal and reliable reproducer/benchmark is going to be the critical piece (that's currently missing) for this to be investigated further.

("I saw this issue too!" is a helpful data point that can be communicated by adding a :+1: reaction to the top or any other post in this issue without making a dedicated comment that otherwise does not get us any closer to a solution. :bow::heart:)