Order of PHP_MODULES on Alpine (musl) causes testing failures

[thirdwheel]

Description

Across multiple PHP versions (7.2.x, 7.3.x, 7,4.x, 8.x), when building on Alpine Linux I have had to edit the Makefile and reorder the libraries in the PHP_MODULES variable so the tests run, otherwise all tests fail because a library relied on a prerequisite that was loaded out of order.

I'm guessing this doesn't occur on distributions that don't use something like musl libc, which might be somewhat maliciously compliant when it comes to handling libraries at runtime like this.

PHP Version

8.1.11

Operating System

Alpine Linux

[Danack]

Please can you give the error message. In particular, which modules are presumably failing to be loaded as they are order dependent on something else.

[Danack]

Also, please can you say how you are installing the extensions. Or even better, provide a github repo containing a docker/docker-compose container that shows the problem.

[thirdwheel]

Please can you give the error message. In particular, which modules are presumably failing to be loaded as they are order dependent on something else.

In this case it's mysqli and pdo_mysql that rely on mysqlnd, but I've also seen it with xml and wddx in 7.2.x (I know it's EOL but might offer a clue as to what's going on here)

Also, please can you say how you are installing the extensions. Or even better, provide a github repo containing a docker/docker-compose container that shows the problem.

Here is my configure options for 8.0.24:

./configure --build=x86_64-linux-gnu --prefix=/opt/php/8.0.24/ --includedir=/opt/php/8.0.24/include --mandir=/opt/php/8.0.24/share/man --infodir=/opt/php/8.0.24/share/info --sysconfdir=/opt/php/8.0.24/etc --localstatedir=/opt/php/8.0.24/var --libdir=/opt/php/8.0.24/lib/x86_64-linux-gnu --libexecdir=/opt/php/8.0.24/lib/x86_64-linux-gnu --enable-cli --enable-cgi --disable-phpdbg --disable-all --disable-debug --disable-rpath --disable-static --with-pic --with-layout=GNU --without-pear --enable-filter --with-openssl=yes --with-password-argon2=/usr --with-mhash=/usr --with-libxml --enable-session --with-sodium --with-zlib=/usr --with-zlib-dir=/usr --with-unixODBC=shared,/usr --with-pdo-odbc=shared,unixODBC,/usr --with-readline=shared,/usr --with-sqlite3=shared,/usr --with-pdo-sqlite=shared,/usr --enable-dom=shared --enable-simplexml=shared --enable-xml=shared --enable-xmlreader=shared --enable-xmlwriter=shared --with-xsl=shared,/usr --with-pdo-dblib=shared,/usr --with-gmp=shared,/usr --with-ldap=shared,/usr --with-ldap-sasl=/usr --enable-calendar=shared --enable-ctype=shared --enable-exif=shared --enable-fileinfo=shared --enable-ftp=shared --with-openssl-dir=/usr --with-gettext=shared,/usr --with-iconv=shared --enable-pdo=shared --enable-phar=shared --enable-posix=shared --enable-shmop=shared --enable-sockets=shared --enable-sysvmsg=shared --enable-sysvsem=shared --enable-sysvshm=shared --enable-tokenizer=shared --enable-intl=shared --with-snmp=shared,/usr --with-pgsql=shared,/usr --with-pdo-pgsql=shared,/usr --enable-mbstring=shared --enable-mbregex --enable-opcache --enable-huge-code-pages --with-imap=shared,/usr --with-kerberos --with-imap-ssl=yes --with-bz2=shared,/usr --enable-mysqlnd=shared --enable-mysqlnd-compression-support --with-mysqli=shared,mysqlnd --with-pdo-mysql=shared,mysqlnd --with-tidy=shared,/usr --enable-soap=shared --enable-dba=shared --with-db4=/usr --with-gdbm --enable-inifile --enable-flatfile --without-pdo-firebird --with-pspell=shared,/usr --enable-bcmath=shared --with-curl=shared,/usr --disable-dtrace build_alias=x86_64-linux-gnu

I also build in a dedicated build folder, in case that makes a difference. I am running a test build, and I will post a log along with my generated Makefile once it completes.

[Danack]

Are you able to reproduce this in a particular Alpine Linux container image?

Please can you give the error message.

Error messages are clearer than descriptions. Please can you provide the error message you're seeing.

[thirdwheel]

Please can you provide the error message you're seeing.

Like I said, I am generating the log which will show the error I'm getting.

I remembered head is a thing. Here's the first 100 lines, can supply full log on request. Makefile is attached.

buildozer:~/php-8.0.24/_testbuildcd$ head -100 maketest.log

Build complete.
Don't forget to run 'make test'.

=====================================================================
PHP         : /home/user.name/php-8.0.24/_testbuildcd/sapi/cli/php
Warning: PHP Startup: Unable to load dynamic library 'mysqli.so' (tried: /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so (Error relocating /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so: mysqlnd_poll: symbol not found), /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so.so (Error loading shared library /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so.so: No such file or directory)) in Unknown on line 0

Warning: PHP Startup: Unable to load dynamic library 'pdo_mysql.so' (tried: /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so (Error relocating /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so: mysqlnd_connection_connect: symbol not found), /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so.so (Error loading shared library /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so.so: No such file or directory)) in Unknown on line 0

PHP_SAPI    : cli
PHP_VERSION : 8.0.24
ZEND_VERSION: 4.0.24
PHP_OS      : Linux - Linux buildozer 5.10.61-0-lts #1-Alpine SMP Fri, 27 Aug 2021 05:29:55 +0000 x86_64
INI actual  : /home/user.name/php-8.0.24/_testbuildcd/tmp-php.ini
More .INIs  :
CWD         : /home/user.name/php-8.0.24
Extra dirs  :
VALGRIND    : Not used
=====================================================================
TIME START 2022-10-13 05:04:20
=====================================================================
PHP Warning:  PHP Startup: Unable to load dynamic library 'mysqli.so' (tried: /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so (Error relocating /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so: mysqlnd_poll: symbol not found), /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so.so (Error loading shared library /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so.so: No such file or directory)) in Unknown on line 0
PHP Warning:  PHP Startup: Unable to load dynamic library 'pdo_mysql.so' (tried: /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so (Error relocating /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so: mysqlnd_connection_connect: symbol not found), /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so.so (Error loading shared library /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so.so: No such file or directory)) in Unknown on line 0
PHP Warning:  PHP Startup: Unable to load dynamic library 'mysqli.so' (tried: /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so (Error relocating /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so: mysqlnd_poll: symbol not found), /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so.so (Error loading shared library /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so.so: No such file or directory)) in Unknown on line 0
PHP Warning:  PHP Startup: Unable to load dynamic library 'pdo_mysql.so' (tried: /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so (Error relocating /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so: mysqlnd_connection_connect: symbol not found), /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so.so (Error loading shared library /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so.so: No such file or directory)) in Unknown on line 0
FAIL phpt EXTENSIONS directive with static module [tests/run-test/bug75042-2.phpt]
PHP Warning:  PHP Startup: Unable to load dynamic library 'mysqli.so' (tried: /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so (Error relocating /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so: mysqlnd_poll: symbol not found), /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so.so (Error loading shared library /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so.so: No such file or directory)) in Unknown on line 0
PHP Warning:  PHP Startup: Unable to load dynamic library 'pdo_mysql.so' (tried: /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so (Error relocating /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so: mysqlnd_connection_connect: symbol not found), /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so.so (Error loading shared library /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so.so: No such file or directory)) in Unknown on line 0
PHP Warning:  PHP Startup: Unable to load dynamic library 'mysqli.so' (tried: /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so (Error relocating /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so: mysqlnd_poll: symbol not found), /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so.so (Error loading shared library /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so.so: No such file or directory)) in Unknown on line 0
PHP Warning:  PHP Startup: Unable to load dynamic library 'pdo_mysql.so' (tried: /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so (Error relocating /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so: mysqlnd_connection_connect: symbol not found), /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so.so (Error loading shared library /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so.so: No such file or directory)) in Unknown on line 0
PASS phpt EXTENSIONS directive with nonexistent shared module [tests/run-test/bug75042-3.phpt]
PHP Warning:  PHP Startup: Unable to load dynamic library 'mysqli.so' (tried: /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so (Error relocating /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so: mysqlnd_poll: symbol not found), /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so.so (Error loading shared library /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so.so: No such file or directory)) in Unknown on line 0
PHP Warning:  PHP Startup: Unable to load dynamic library 'pdo_mysql.so' (tried: /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so (Error relocating /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so: mysqlnd_connection_connect: symbol not found), /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so.so (Error loading shared library /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so.so: No such file or directory)) in Unknown on line 0
PHP Warning:  PHP Startup: Unable to load dynamic library 'mysqli.so' (tried: /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so (Error relocating /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so: mysqlnd_poll: symbol not found), /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so.so (Error loading shared library /home/user.name/php-8.0.24/_testbuildcd/modules/mysqli.so.so: No such file or directory)) in Unknown on line 0
PHP Warning:  PHP Startup: Unable to load dynamic library 'pdo_mysql.so' (tried: /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so (Error relocating /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so: mysqlnd_connection_connect: symbol not found), /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so.so (Error loading shared library /home/user.name/php-8.0.24/_testbuildcd/modules/pdo_mysql.so.so: No such file or directory)) in Unknown on line 0
SKIP phpt EXTENSIONS directive with shared module [tests/run-test/bug75042.phpt] reason: openssl is built static
FAIL EXPECT [tests/run-test/test001.phpt]
FAIL EXPECTF [tests/run-test/test002.phpt]
FAIL EXPECTREGEX [tests/run-test/test003.phpt]
FAIL INI section allows '=' [tests/run-test/test004.phpt]
PASS Error messages are shown [tests/run-test/test006.phpt]
FAIL dirname test [tests/run-test/test007.phpt]
FAIL print_r(Object) [tests/run-test/test009.phpt]
FAIL STDIN input [tests/run-test/test010.phpt]
FAIL EXPECT_EXTERNAL [tests/run-test/test011.phpt]
FAIL EXPECTF_EXTERNAL [tests/run-test/test012.phpt]
FAIL EXPECTREGEX_EXTERNAL [tests/run-test/test013.phpt]
FAIL Trivial "Hello World" test [tests/basic/001.phpt]
FAIL Simple POST Method test [tests/basic/002.phpt]
FAIL GET and POST Method combined [tests/basic/003.phpt]
FAIL Two variables in POST data [tests/basic/004.phpt]
FAIL Three variables in POST data [tests/basic/005.phpt]
FAIL Add 3 variables together and print result [tests/basic/006.phpt]
FAIL Multiply 3 variables and print result [tests/basic/007.phpt]
FAIL Divide 3 variables and print result [tests/basic/008.phpt]
FAIL Subtract 3 variables and print result [tests/basic/009.phpt]
FAIL Testing | and & operators [tests/basic/010.phpt]
FAIL Testing $argc and $argv handling (GET) [tests/basic/011.phpt]
FAIL Testing $argc and $argv handling (cli) [tests/basic/012.phpt]
FAIL POST Method test and arrays [tests/basic/013.phpt]
FAIL POST Method test and arrays - 2 [tests/basic/014.phpt]
FAIL POST Method test and arrays - 3 [tests/basic/015.phpt]
FAIL POST Method test and arrays - 4 [tests/basic/016.phpt]
FAIL POST Method test and arrays - 5 [tests/basic/017.phpt]
FAIL POST Method test and arrays - 6 [tests/basic/018.phpt]
FAIL POST Method test and arrays - 7 [tests/basic/019.phpt]
FAIL POST Method test and arrays - 8 [tests/basic/020.phpt]
FAIL Bug #37276 (problems witch $_POST array) [tests/basic/021.phpt]
FAIL Cookies test#1 [tests/basic/022.phpt]
FAIL Cookies test#2 [tests/basic/023.phpt]
FAIL Test HTTP_RAW_POST_DATA with excessive post length [tests/basic/025.phpt]
FAIL RFC1867 character quotting [tests/basic/028.phpt]
FAIL Shift_JIS request [tests/basic/029.phpt]
FAIL Bug#55504 (Content-Type header is not parsed correctly on HTTP POST request) [tests/basic/030.phpt]
FAIL Bug#55504 (Content-Type header is not parsed correctly on HTTP POST request) [tests/basic/031.phpt]
FAIL Bug#18792 (no form variables after multipart/form-data) [tests/basic/032.phpt]
FAIL Bug #20539 (PHP CLI Segmentation Fault) [tests/basic/bug20539.phpt]
FAIL Bug #29971 (variables_order behaviour) [tests/basic/bug29971.phpt]
FAIL Bug #31875 get_defined_functions() should not list disabled functions [tests/basic/bug31875.phpt]
FAIL Bug #45986 (wrong error message for a non existent file on rename) [tests/basic/bug45986.phpt]
FAIL Bug #53180 (post_max_size=0 partly not working) [tests/basic/bug53180.phpt]
FAIL Req #54514 (Get php binary path during script execution) [tests/basic/bug54514.phpt]
FAIL Bug #55500 (Corrupted $_FILES indices lead to security concern) [tests/basic/bug55500.phpt]
FAIL Bug #61000 (Exceeding max nesting level doesn't delete numerical vars) [tests/basic/bug61000.phpt]
FAIL php://input is empty when enable_post_data_reading=Off [tests/basic/bug67198.phpt]
FAIL Bug #67988 (htmlspecialchars() does not respect default_charset specified by ini_set) [tests/basic/bug67988.phpt]
FAIL Bug #71273 A wrong ext directory setup in php.ini leads to crash [tests/basic/bug71273.phpt]
FAIL Bug #73969: segfault on debug_print_backtrace with require() call [tests/basic/bug73969.phpt]
FAIL Bug #78236 (convert error on receiving variables when duplicate [) [tests/basic/bug78236.phpt]
FAIL Bug #78929 (plus signs in cookie values are converted to spaces) [tests/basic/bug78929.phpt]
FAIL Cookies Security Bug [tests/basic/bug79699.phpt]
FAIL Bug #80384 large reads cause filters to internally buffer large amounts of memory [tests/basic/bug80384.phpt]
FAIL Test that float to string and string to float casts are consistent [tests/basic/consistent_float_string_casts.phpt]
FAIL enable_post_data_reading: basic test [tests/basic/enable_post_data_reading_01.phpt]
FAIL enable_post_data_reading: rfc1867 [tests/basic/enable_post_data_reading_02.phpt]
FAIL enable_post_data_reading: always_populate_raw_post_data has no effect (1) [tests/basic/enable_post_data_reading_03.phpt]
FAIL enable_post_data_reading: always_populate_raw_post_data has no effect (2) [tests/basic/enable_post_data_reading_04.phpt]
FAIL enable_post_data_reading: using multiple input streams [tests/basic/enable_post_data_reading_05.phpt]

Are you able to reproduce this in a particular Alpine Linux container image?

I'm not well versed enough in containers to be able to do that.

[thirdwheel]

Have supplied an edited comment with details that might help.

[Danack]

Just in case it matters, can you say which version of Alpine you're using, which should be displayable with either lsb_release -a or alpine -version?

[Danack]

To note for others, PHP_MODULES is a m4/configure generated thing.

[Danack]

btw this seems a similar message https://github.com/codecasts/php-alpine/issues/54

[thirdwheel]

Just in case it matters, can you say which version of Alpine you're using, which should be displayable with either lsb_release -a or alpine -version?

I don't have either of those commands but I can show you the contents of /etc/alpine-release:

buildozer:~/php-8.0.24/_testbuildcd$ cat /etc/alpine-release
3.14.2

[thirdwheel]

btw this seems a similar message codecasts/php-alpine#54

A key difference here is that this occurs in runtime - my issue occurs when trying to run the tests.

[thirdwheel]

Also I updated the title because it isn't a build failure.

[thirdwheel]

I just tried on a fresh install of Alpine Linux 3.16.2 and got the same result.

[Danack]

A key difference here is that this occurs in runtime - my issue occurs when trying to run the tests.

The tests are run through PHP.

[thirdwheel]

A key difference here is that this occurs in runtime - my issue occurs when trying to run the tests.

The tests are run through PHP.

Fair point. It's also got the same cause - libraries loaded out of order. Could this be an issue caused by musl and how it handles libraries, rather than anything that can/should be done in PHP itself? And if so, will I just have to soldier on with editing the Makefile to run the tests?

[Danack]

Could this be an issue caused by musl

It could be many things. I personally wouldn't like to guess.

If you want someone to investigate it, finding or providing an Alpine Linux container that has the required libraries installed to compile PHP would allow someone else to investigate it.

There are quite a few people who provide PHP on alpine...finding out where their build container is might be good useful. I'm personally not going to spend a few hours trying to get a PHP build environment on alpine working.

[thirdwheel]

I followed a guide on creating a Dockerfile and adapted it to this use case. It's my first proper go at this, so please be brutal in your criticisms.

https://gist.githubusercontent.com/thirdwheel/057af38cf8384dd3184cea8a28a9dd8a/raw/Dockerfile

Use it to build the image, run it, exec into sh and run the build.sh script with a PHP version (word of warning, it will use all available processors for the build, so don't run this on anything where CPU time is scarce):

./build.sh 8.1.11

Once it builds, make test to show the error. I have installed nano so you can see how moving mysqlnd.la to the start of the PHP_MODULES variable allows the tests to run.

[Danack]

This issue isn't limited to Alpine. I can reproduce on OSX with:

./configure \
--without-iconv \
--without-pcre-jit \
--without-pear \
--without-sqlite3 \
--without-pdo-sqlite \
--enable-mysqlnd=shared \
--with-mysqli=shared,mysqlnd \
--with-pdo-mysql=shared,mysqlnd

The issue is when building all of mysqlnd, mysqli and pdo_mysql as shared extensions, the loading order matters. Forcing mysqlnd to be loaded before the others can be done for tests. Not sure how to do it for installed PHP.

[thirdwheel]

Forcing mysqlnd to be loaded before the others can be done for tests. Not sure how to do it for installed PHP.

For runtime, it's a configuration issue with a documented solution already. Just like how mbstring needs to be loaded before exif, mysqlnd will need to be loaded before mysqli/pdo_mysql.

The issue at hand is that the tests fail when mysqlnd is built as a shared extension.