PHP 7.4 FPM Apache - apt-get install error - "no installation candidate"

[mlantz7]

With the latest changes, my install no longer works.

It gets stuck on the apt-get line, with "no installation candidate" for both php7.4-dev and php-pear

Am I going to need to change something, or is this a bug?

---

FROM shinsenter/phpfpm-apache:php7.4

# Update and install packages
RUN apt-get update && apt-get install -y libmcrypt-dev php7.4-dev php-pear

# Install mcrypt via PECL
RUN pecl config-set php_ini "${PHP_INI_DIR}/php.ini"
RUN pecl install mcrypt

# And and Enable BZ2 and Mcrypt extensions
RUN phpaddmod bz2 && phpenmod bz2 mcrypt

# Enable RemoteIP to get proper client IP when Cloudflare and/or AWS Load Balancer is present
RUN a2enconf remoteip
RUN echo "RemoteIPHeader X-Forwarded-For" > /etc/apache2/conf-available/remoteip.conf

# Control your timezone
ENV TZ="America/Chicago"

# sets GID and UID
ENV PUID=33
ENV PGID=33

# Sets the directory from which Apache will serve files
ENV WEBHOME="/srv/www"
ENV FIX_WEBHOME_PERMISSION="true"

# Set Apache root folder within $WEBHOME
ENV APACHE_DOCUMENT_ROOT="/public_html"

# Set root path of what PHP is allowed to access (to get access to the "data" folder from EFS)
ENV PHP_OPEN_BASEDIR="/srv"

# Copy source to the right place and setup proper ownership
COPY . /srv/www/

WORKDIR $WEBHOME

[shinsenter]

@mlantz7

New docker images are built based on official php docker images, so basically the OS of docker images will become debian or alpine instead of Ubuntu like old docker images.

With new docker images, you also do not necessarily have to install additional packages to build php extensions you need, just simply use the phpaddmod utility, it will help you download necessary packages as well as remove unnecessary packages after successfully installing extensions.

Please try building your docker image by using the following Dockerfile:

FROM shinsenter/php:7.4-fpm-apache

# Just call phpaddmod for installing PHP extensions
# it will take care of all required steps to compile and activate the extensions.
RUN phpaddmod bz2 mcrypt

# To enable Apache modules, please use `a2enmod`, not `a2enconf`
# By the way, mode_remoteip and X-Forwarded-For are already enabled by default
# You do not need to run the next 3 lines
# RUN a2enmod remoteip
# RUN mkdir -p /etc/apache2/conf-enabled
# RUN echo "RemoteIPHeader X-Forwarded-For" > /etc/apache2/conf-enabled/remoteip.conf

# Control your timezone
ENV TZ="America/Chicago"

# PUIG and PGID are renamed to $APP_UID and $APP_GID
# You also do not need to set uid and gid, just keep OS defaults
# But it is OK to override uid and gid
ENV APP_UID=33
ENV APP_GID=33

# Application path is now $APP_PATH
ENV APP_PATH="/srv/www"

# Web document root is now $DOCUMENT_ROOT
ENV DOCUMENT_ROOT="/public_html"

# Copy source with right ownership
COPY --chown=www-data:www-data . /srv/www/

WORKDIR $APP_PATH

Test build:

docker build -f Dockerfile . -t test

Test PHP modules

docker run --rm test php -m

Results:

[PHP Modules]
apcu
bcmath
bz2
calendar
Core
ctype
curl
date
dom
exif
fileinfo
filter
ftp
gd
gettext
hash
iconv
igbinary
imap
intl
json
libxml
mbstring
mcrypt
mysqli
mysqlnd
openssl
pcntl
pcre
PDO
pdo_mysql
pdo_pgsql
pdo_sqlite
Phar
posix
readline
redis
Reflection
session
SimpleXML
sodium
SPL
sqlite3
standard
tokenizer
uuid
xml
xmlreader
xmlwriter
yaml
Zend OPcache
zip
zlib

[Zend Modules]
Zend OPcache

I am working on the new documentation. I hope this helps. https://github.com/shinsenter/php/blob/5.x/README.md

Best regards

[mlantz7]

Thanks for the reply and info.

So I am now having issues with the document root and app path it seems. Setting them as you mention does not seem to work.

My files are all in the right place (/srv/www) but when I check the container its still set to /var/www/html so trying to access the site gets 404 errors. Any ideas on that?

[mlantz7]

BTW I am definitely not a super-expert in Docker or anything...

We do use this in our production environment (via Amazon ECS), so these breaking changes are definitely a pain for us as we can't deploy new code without fixing and re-testing everything.

Is there not a way we can use the non-5.x (ie main) branch when pulling from Docker hub? That way we could continue with what has been working until we are able to implement and test the 5.x version?

[mlantz7]

OK I think I figured out the previous issue - it was using the wrong shinsenter container...

Does this still automatically run Composer or is that something we will have to do now? if so, what is the correct way to do that?

[shinsenter]

@mlantz7

I apologize for the changes coming from the new docker images that may affect your current project. We are considering using different docker image tags to maintain the old docker image versions this week.

The new docker images are built from the official php docker images, ensuring packages receive the latest bug fixes from upstream compared to the old image versions. I hope you can understand and be patient, and we are trying to minimize the impact on projects using the old docker image versions.

Regarding your issue, I understand it may be difficult for you to migrate your containers for production, however I think it should not be too complex if you don't have too many customizations from the base image.

If you don't mind sharing with me the setup for your project, I think I may be able to assist you in migrating your docker container to work with the new docker images.

Best regards

[mlantz7]

There is really not much more to share aside from the Dockerfile I shared earlier. That is almost exactly what is run to create our ECS containers.

The issue I seem to have right now is with Composer. I see its still installed, but now I get this error:

root@2059b7e5746e:/srv/www# composer install
[29-Jan-2024 12:09:19 America/Chicago] PHP Warning:  Phar::mapPhar(): open_basedir restriction in effect. File(/usr/local/bin/composer) is not within the allowed path(s): (/srv) in /usr/local/bin/composer on line 28
[29-Jan-2024 12:09:19 America/Chicago] PHP Warning:  require(phar://composer.phar/bin/composer): failed to open stream: phar error: invalid url or non-existent phar "phar://composer.phar/bin/composer" in /usr/local/bin/composer on line 29
[29-Jan-2024 12:09:19 America/Chicago] PHP Fatal error:  require(): Failed opening required 'phar://composer.phar/bin/composer' (include_path='.:/usr/local/lib/php') in /usr/local/bin/composer on line 29

I am setting PHP_OPEN_BASEDIR=/srv as I did before, but it seems like it doesn't like that anymore. Is there something wrong with how I am doing it?

I do it this way because we also mount an EFS volume at /srv/data that the app needs access to.

[mlantz7]

So I went ahead and just commented out the PHP_OPEN_BASEDIR line to test things. Not sure what effect it will have, but even when I do that and try to run "composer install", I get a new error:

root@46472c39c650:/srv/www# composer install
Installing dependencies from lock file (including require-dev)
Verifying lock file contents can be installed on current platform.
Your lock file does not contain a compatible set of packages. Please run composer update.

  Problem 1
    - wrench/wrench is locked to version v2.0.11 and an update of this package was not requested.
    - wrench/wrench v2.0.11 requires ext-sockets * -> it is missing from your system. Install or enable PHP's sockets extension.
  Problem 2
    - wrench/wrench v2.0.11 requires ext-sockets * -> it is missing from your system. Install or enable PHP's sockets extension.
    - chrome-php/chrome v0.8.1 requires wrench/wrench ^2.0 -> satisfiable by wrench/wrench[v2.0.11].
    - chrome-php/chrome is locked to version v0.8.1 and an update of this package was not requested.

Seems like its say the "sockets" extension is not present but is needed. I didn't explicitly install it before, so any idea what is going on there?

[shinsenter]

There is really not much more to share aside from the Dockerfile I shared earlier. That is almost exactly what is run to create our ECS containers.

The issue I seem to have right now is with Composer. I see its still installed, but now I get this error:

root@2059b7e5746e:/srv/www# composer install
[29-Jan-2024 12:09:19 America/Chicago] PHP Warning:  Phar::mapPhar(): open_basedir restriction in effect. File(/usr/local/bin/composer) is not within the allowed path(s): (/srv) in /usr/local/bin/composer on line 28
[29-Jan-2024 12:09:19 America/Chicago] PHP Warning:  require(phar://composer.phar/bin/composer): failed to open stream: phar error: invalid url or non-existent phar "phar://composer.phar/bin/composer" in /usr/local/bin/composer on line 29
[29-Jan-2024 12:09:19 America/Chicago] PHP Fatal error:  require(): Failed opening required 'phar://composer.phar/bin/composer' (include_path='.:/usr/local/lib/php') in /usr/local/bin/composer on line 29

I am setting PHP_OPEN_BASEDIR=/srv as I did before, but it seems like it doesn't like that anymore. Is there something wrong with how I am doing it?

I do it this way because we also mount an EFS volume at /srv/data that the app needs access to.

@mlantz7

PHP has a configuration option called open_basedir that restricts which directories a PHP script can access. This option helps improve security by preventing PHP from accessing sensitive files/directories.

In previous docker images, setting open_basedir did not work as expected. Now it will work exactly as described in the official PHP documentation: https://www.php.net/manual/en/ini.core.php#ini.open-basedir

Meaning if you set open_basedir to "/srv", PHP will only allow accessing files/directories under /srv. The issue here is that PHP applications like Composer are installed outside of /srv so they are restricted by open_basedir.

The document also says:

open_basedir is just an extra safety net, that is in no way comprehensive, and can therefore not be relied upon when security is needed.

So, setting PHP_OPEN_BASEDIR in your project is optional, and I think you can choose omitting that settings if it becomes your troubles.

To resolve the problem with PHP_OPEN_BASEDIR, you should temporarily comment out the PHP_OPEN_BASEDIR line, or add to PHP_OPEN_BASEDIR all of the paths your application may later require access to (which is often more than just /srv, and include /tmp, /var/log etc depending on your framework). For example:

ENV PHP_OPEN_BASEDIR=/srv:/tmp:/usr/local/bin

This will allow PHP to access /srv, /tmp and /usr/local/bin.

[shinsenter]

So I went ahead and just commented out the PHP_OPEN_BASEDIR line to test things. Not sure what effect it will have, but even when I do that and try to run "composer install", I get a new error:

root@46472c39c650:/srv/www# composer install
Installing dependencies from lock file (including require-dev)
Verifying lock file contents can be installed on current platform.
Your lock file does not contain a compatible set of packages. Please run composer update.

  Problem 1
    - wrench/wrench is locked to version v2.0.11 and an update of this package was not requested.
    - wrench/wrench v2.0.11 requires ext-sockets * -> it is missing from your system. Install or enable PHP's sockets extension.
  Problem 2
    - wrench/wrench v2.0.11 requires ext-sockets * -> it is missing from your system. Install or enable PHP's sockets extension.
    - chrome-php/chrome v0.8.1 requires wrench/wrench ^2.0 -> satisfiable by wrench/wrench[v2.0.11].
    - chrome-php/chrome is locked to version v0.8.1 and an update of this package was not requested.

Seems like its say the "sockets" extension is not present but is needed. I didn't explicitly install it before, so any idea what is going on there?

@mlantz7

As the error message itself pointed out, there are two things you need to do to fix this.

First, since the official docker images do not include the sockets extension by default, you need to install it in your Dockerfile using the phpaddmod utility I mentioned earlier.

RUN phpaddmod bz2 mcrypt sockets

The reason it did not happen on previous docker images is that those images used an unofficial source to install php on Ubuntu, and they had way too many things built in by default, most of which are useless for many different projects that do not need those pre-built extensions. It also poses higher security risks since they are not built from official sources.

Second, you need to delete the composer.lock file in your project directory, and rerun composer update to regenerate the composer.lock file. I think that should perfectly fix everything.

COPY --chown=www-data:www-data . /srv/www/

WORKDIR $APP_PATH

RUN composer update

The reason is that the missing sockets extension causes composer to generate an invalid composer.lock file initially. By deleting it and re-generating it, you ensure the lock file matches the current set of extensions available in your docker image.

[mlantz7]

I've been able to figure most things out so i think we are good to go here. Thanks for helping me work through it!

[mlantz7]

So I ran into another issue...

It seems that RemoteIP is not working as expected. You mentioned that it was automatically enabled, but I am getting the IP of the AWS load balancer now instead of the end user's IP like I used to.

I see the config setup in the container seems to be quite different, without all of the usual apache folders like conf-available, conf-enabled, etc...

I see a folder "docker.d" that has mpm-tuning.conf?

I do see in apache2.conf this:

<IfModule remoteip_module>
    RemoteIPHeader X-Forwarded-For
    RemoteIPTrustedProxy 10.0.0.0/8
    RemoteIPTrustedProxy 172.16.0.0/12
    RemoteIPTrustedProxy 192.168.0.0/16
    RemoteIPTrustedProxy 169.254.0.0/16
    RemoteIPTrustedProxy 127.0.0.0/8
    RemoteIPTrustedProxy fd00::/8
</IfModule>

So now I see the issue - RemoteIPTrustedProxy does not have the needed ranges for my AWS load balancer.

Previously when I setup the conf, I didn't have any RemoteIPTrustedProxy entries, so it worked for all.

We also use CloudFlare which has a ton of IP ranges as well.

With this new setup, what would be the best way to override this to work as desired?

[mlantz7]

So I went ahead and created a new conf file and put it in docker.d. Seems to do the trick.

So now I have something else... :)

I got this running in my part of my ECS production env, and performance seems to be MUCH worse than the last version before all of these changes.

Do you have any idea on what may have changed that would cause performance to suffer? Its a pretty big difference - server load and CPU are much higher than before.

Thanks again for your help!

[shinsenter]

Hi @mlantz7

Thank you again for using and providing feedback on our docker images.

Regarding your issue, we have pushed a bug-fix related to retrieving the client IP address and hope new docker images with the fix will be updated soon within the next 5-6 hours. We look forward to continue receiving your feedback on the bug-fix release.

We also commented out those RemoteIPTrustedProxy lines, and you can add yours if needed.

Regarding the performance issue you mentioned, I need specific measurement details to determine the cause. Apologies that I cannot draw any convincing conclusions about this issue, so we hope to address it by adding older docker image versions under different tag names, and will contact you when they are published.

[mlantz7]

Thanks for the reply.

I am wondering if the performance issue is related to the "tuning" being done for both php-fpm and apache...

What were the old values being used for those in the previous (non-5.x) branch?

What I think may be happening is that its using incorrect values because of how Amazon ECS works. My containers are set to 1 CPU and 2GB of RAM, but I see they are being reported as 2 CPUs and 8MB of RAM.

I see that I can manually set the Apache settings, but it doesn't seem like I can set the PHP-FPM ones?

Anyway, if I can determine what old default settings were, that would help.

Here is an image from NewRelic that is pretty telling. The vertical line shows when I deployed the new code... and before it was only ever running 1 task, and now it runs 2-3 to try and keep up. So yeah pretty serious issue! :)

[shinsenter]

Hi @mlantz7 Auto tuning feature is now disabled by default. Thank you for the information.

[mlantz7]

But I believe that only currently applies to Apache right? Because there are not settings for php that can be changed at all?

SEE HERE:

https://github.com/shinsenter/php/blob/de2251cc60d430d7c0097ba555e15ffa341ae327/src/php/common/shell-php/fpm-pm

I found the old branch used "ondemand" instead of "dynamic" as well, and again there is no way to change this now.

Let me know if I am missing something.

[shinsenter]

Hi @mlantz7

You are absolutely correct.

Setting pm = ondemand has been configured in previous docker image versions and it appears to work better in various situations. I have changed the default back from dynamic to ondemand, and also disabled auto-tuning for both PHP-FPM and Apache by default (you can easily re-enable auto-tuning by setting the environment variables ENABLE_TUNING_FPM=1 and ENABLE_TUNING_MPM=1 for your container).

I've added this information to the README as well.

You can run this to view all configurable environment variables for PHP settings:

docker run shinsenter/php:7.4-fpm-apache env | grep PHP_ | sort

These changes are scheduled to be reflected into new docker images in a few hours, so please be patient. https://github.com/shinsenter/php/actions

[mlantz7]

Looks great, thank you for the updates.

I took a look at the environment variables...

I see this in /usr/local/etc/php-fpm.d/zz-generated-settings.conf:

pm = ${PHP_PM}
pm.max_children = ${PHP_PM_MAX_CHILDREN}
pm.max_requests = ${PHP_PM_MAX_REQUESTS}
pm.max_spare_servers = ${PHP_PM_MAX_SPARE_SERVERS}
pm.min_spare_servers = ${PHP_PM_MIN_SPARE_SERVERS}
pm.process_idle_timeout = ${PHP_PM_PROCESS_IDLE_TIMEOUT}
pm.start_servers = ${PHP_PM_START_SERVERS}
pm.status_path = ${PHP_PM_STATUS_PATH}

But I don't see defaults being set for many of them, even though I see values when running the env command you shared above? How does that work?

[shinsenter]

@mlantz7

The default values for the environment variables needed for my docker images to function are currently defined in the /etc/docker-envvars file.

Feel free to refer to the contents of that file.

cat /etc/docker-envvars

[mlantz7]

Ok I see now. Thanks!

I look forward to testing this more in the morning - hopefully this takes care of the issues I was seeing.

[shinsenter]

@mlantz7 It seems like the docker images for the version you need have been updated. Please help me test whether it has resolved your issue.

[mlantz7]

So I started using the new image, and still running into performance issues.

In looking at things, I started to think it was because of PHP session garbage collection (I think in the old image session.gc_probability=0 - it is 1 in the new image), but made an update and it still seemed to keep heavy load. I will be doing some additional tests.

If its easy enough for you, can you run phpinfo() in both the old (pre-5.x) image and in the new one and share what is returned. I think comparing all of the config values may help figure out what is going on?

FYI one other thing I ran into was "igbinary" being the default for session serialization. Not sure if that would cause an issue for anyone else, but it was a change so caused errors with our sessions. I just changed it back to using "php" instead and its all good.

[mlantz7]

So I went through all of the PHP ini variables, and everything seems fine and I can't see where there would be more performance issues aside from me needing to turn off the session garbage collection.

However, after additional testing, I am still seeing issues. So far they seem connected to running PHP "cli" tasks, which we run via exec_cmd() to process things in the background in some cases when certain requests come in. Our test case uses this heavily. Can you think of anything that could cause the PHP cli to perform poorly, and FPM work fine?

In any case, I will continue looking into and testing different things. Since we had zero performance issues with the previous image version, there has to be something going on...

[shinsenter]

@mlantz7

Thank you for providing insights and suggestions to improve the efficiency of the docker images, I really appreciate it.

I want to let you know that all old docker images have been published with *-deprecated tag names. https://hub.docker.com/r/shinsenter/php/tags?page=1&name=-deprecated https://hub.docker.com/r/shinsenter/phpfpm-apache/tags?page=1&name=-deprecated

You now can revert back to using the old versions with the settings you previously used. However, we will no longer provide any update for the old docker images, please understand.

If you decide to use new docker images, I hope you can provide more feedback so that we can improve them better.

Regarding your feedback, I have decided to keep the default values for php.ini directives, and will only consider customizing some values related to logging and basic security, hoping it will somehow improve the performance you expect. The default values for php.ini directives are defined here: https://github.com/shinsenter/php/blob/5.x/src/php/common/php-ini-directives.dockerfile

Regarding executing PHP CLI commands using exec_cmd() (did you mean PHP's exec() or shell_exec()?), I am not sure which programs you have executed so I cannot give any conclusion about the issue. Please provide more information.

New docker images with these changes will be updated in a few hours, please be patient. https://github.com/shinsenter/php/actions

[mlantz7]

Thanks for the details.

On the exec_cmd() comment, sorry that is an internal function name. Its a wrapper for PHP exec() function.

Anyway, I am just calling something like exec('php myfile.php') from it, which is a common occurrence in our app. I just noticed when testing that the process list (using top) got really backed up with these types of calls to PHP.

I am just wondering if there could be any settings in the image that could cause issues with running PHP in its CLI form that is different from the last image? I am not too familiar with that, and how it differs from before. I think previously php-cli had its on folders and config, etc...

Any insight on that?

[mlantz7]

Also is there not an easy way to remove PHP extensions? I see the "imap" extension is newly added to your image, and I have read of potential performance issues. And since I don't need it, I was thinking to remove it for testing.

[shinsenter]

@mlantz7

Currently, there is no properly way described in the official documentation about removing a PHP extension (including related binaries). However, I think you can temporarily disable an extension by commenting out the line extension=<name>.so in php.ini or PHP config directories.

I think I shall add a utility called phpdelmod to do this. I will also consider later which default extensions should be bundled in docker images.

[mlantz7]

Thanks for the details.

On the exec_cmd() comment, sorry that is an internal function name. Its a wrapper for PHP exec() function.

Anyway, I am just calling something like exec('php myfile.php') from it, which is a common occurrence in our app. I just noticed when testing that the process list (using top) got really backed up with these types of calls to PHP.

I am just wondering if there could be any settings in the image that could cause issues with running PHP in its CLI form that is different from the last image? I am not too familiar with that, and how it differs from before. I think previously php-cli had its on folders and config, etc...

Any insight on that?

I am still looking for more info on this...

I removed some unused extensions and that didn't really help. Whatever is going on is causing this PHP scripts run via exec() to go very slowly at times, with the overhead definitely being in the PHP execution.

Can you think of anything that could help me here?

[shinsenter]

@mlantz7

I appreciate your efforts to find a solution for improving the performance of your project using my new docker images. Unfortunately I have not yet determined the primary cause of the performance impact on your project.

I would greatly appreciate if you could provide a minimal reproducible example that replicates the issue, along with details on how you are evaluating project performance.

As a suggestion, you could try replacing the php command in exec('php myfile.php') with php -n (to execute php without loading php.ini settings), and rerun your application to see if a php.ini setting may be contributing to the performance problem.

Please let me know your findings so I can make an informed guess as to the cause. I look forward to working with you to resolve this issue.

[mlantz7]

I figured it out...

In the new image, opcache.enable_cli=1, and previously it was 0.

Turning off opcache for CLI commands is definitely recommended due to the overhead of creating the cache which is then lost after every run because its stored in process memory, and each run is a new process.

I recommend changing the default back to 0 for your images to prevent issues. Further tweaks could be made by someone if they really wanted to the opcache for CLI processes, which basically requires files to be used instead of memory, which has its own tradeoffs.

[shinsenter]

@mlantz7 Thank you for the information.

Have you tried setting ENV PHP_OPCACHE_ENABLE_CLI=0 and testing again to see if it really improves the performance of your project?

I will look into more information about the performance degradation when enabling opcache for PHP CLI before applying it to docker images.

[mlantz7]

Yes I tested and it improved greatly. I guess you didn't see the screenshot I shared in my last comment?

It shows the clear improvement. The script when from average of 550ms time to now under 200ms, with also huge reduction in CPU usage on the container, as well as reduced memory.

This was definitely the thing I was looking for and clear difference from the previous image.

[shinsenter]

@mlantz7

Apologies, I viewed your message via email this morning and I could not see that there was an image in your message.

I have removed the default setting PHP_OPCACHE_ENABLE_CLI=1 in the php-ini-directives.dockerfile. New docker images will be updated in the next few hours.

Thank you again for the helpful feedback.

[mlantz7]

Glad we could figure it out. Thanks for your work on this package! :)

Do you anticipate introducing any further breaking changes into the image? I want to be able to rely on this in production if possible.

Also, can you give me some insight on the "tidy" build? I know its a lot smaller, but what are the actual differences there?

[shinsenter]

@mlantz7

If the official php docker image does not have any major breaking changes, I expect there will be very few sudden changes in my docker images going forward. However, I'll be making some custom tweaks to optimize build speed, shrink image size, etc. I'm looking to improve these images based on feedback from active users like you. I'll also add a CHANGELOG, though honestly most people ignore those.

The "-tidy" tags are same as the "-alpine" tags. They are built on Alpine Linux so they're super lean. I've tried to keep the Debian and Alpine configs as similar as possible, so folks can swap between them easily. A tiny docker image speeds up CI/CD for some projects.

I also added in a phpdelmod utility to temporarily disable extensions, without deleting the actual binaries. Could come in handy sometimes.

Let me know if you have any other questions!

[shinsenter]

@mlantz7 As your issue seems to have been resolved, I'm going to close this issue. If you have any other problems with the docker images, please open a new issue.