search

mongodb / mongo-php-driver

The Official MongoDB PHP driver
https://pecl.php.net/package/mongodb
Apache License 2.0
888 stars 202 forks source link

Bug & Question: How to compile mongodb statically into PHP? #1549

Closed crazywhalecc closed 4 months ago

crazywhalecc commented 5 months ago

Bug Report

I'm trying to statically compile mongodb with PHP, with following steps:

  1. Download php-src and mongo-php-driver
  2. Extract mongo-php-driver to php-src/ext/mongodb
  3. ./buildconf --force
  4. ./configure --disable-shared --enable-static --disable-phpdbg --enable-cli --enable-mongodb --with-mongodb-system-libs=no --with-mongodb-client-side-encryption=no --with-mongodb-sasl=no --without-iconv

And it shows error:

checking whether to enable MongoDB support... yes
checking for pkg-config... (cached) /opt/homebrew/bin/pkg-config
checking pkg-config is at least version 0.9.0... yes
checking PHP version... configure: error: php-config not found

From mongo source, I found it uses php-config directly to get PHP version, and inline build (static compilation) apparently have no any dev command. So I tried to patch config.m4 like this:

// replaceFileStr is just `file_get_contents`, `str_replace`, `file_put_contents`
// getPHPVersion() assume it `8.2.18`
// getPHPVersionID assume it `80218`
FileSystem::replaceFileStr(SOURCE_PATH . '/php-src/ext/mongodb/config.m4', 'if test -z "$PHP_CONFIG"; then', 'if false; then');
FileSystem::replaceFileStr(SOURCE_PATH . '/php-src/ext/mongodb/config.m4', 'PHP_MONGODB_PHP_VERSION=`${PHP_CONFIG} --version`', 'PHP_MONGODB_PHP_VERSION=' . $this->builder->getPHPVersion());
FileSystem::replaceFileStr(SOURCE_PATH . '/php-src/ext/mongodb/config.m4', 'PHP_MONGODB_PHP_VERSION_ID=`${PHP_CONFIG} --vernum`', 'PHP_MONGODB_PHP_VERSION_ID=' . $this->builder->getPHPVersionID());

And in order to add and test other flags more easily, I directly use static-php-cli to compile it. The main compilation methods and commands have not changed. In short, it uses commands below:

[06:02:15] [INFO] Extension [mongodb] patched before buildconf
[06:02:15] [INFO] Entering dir: /Users/jerry/project/git-project/static-php-cli/source/php-src
[06:02:15] [INFO] [EXEC] ./buildconf --force
[06:02:17] [INFO] Entering dir: /Users/jerry/project/git-project/static-php-cli/source/php-src
[06:02:17] [INFO] mongodb is using  --enable-mongodb  --with-mongodb-system-libs=no --with-mongodb-client-side-encryption=no  --with-mongodb-sasl=no   --with-mongodb-icu=no  --with-mongodb-zstd=no  --with-mongodb-zlib=bundled 
[06:02:17] [INFO] [EXEC] ./configure --prefix= --with-valgrind=no --enable-shared=no --enable-static=yes --disable-all --disable-cgi --disable-phpdbg --enable-cli --disable-fpm --disable-embed --disable-micro --enable-mongodb  --with-mongodb-system-libs=no --with-mongodb-client-side-encryption=no  --with-mongodb-sasl=no   --with-mongodb-icu=no  --with-mongodb-zstd=no  --with-mongodb-zlib=bundled CFLAGS='--target=arm64-apple-darwin -Werror=unknown-warning-option' CPPFLAGS='-I/Users/jerry/project/git-project/static-php-cli/buildroot/include' LDFLAGS='-L/Users/jerry/project/git-project/static-php-cli/buildroot/lib'
[06:02:35] [INFO] [EXEC] make clean
[06:02:36] [INFO] building cli
[06:02:36] [INFO] Entering dir: /Users/jerry/project/git-project/static-php-cli/source/php-src
[06:02:36] [INFO] [EXEC] make EXTRA_CFLAGS='-g -Os -Wimplicit-function-declaration' EXTRA_LIBS=' -lresolv' cli

And finally got this strange error from standard ext:

/Users/jerry/project/git-project/static-php-cli/source/php-src/ext/standard/dir.c:283:8: error: call to undeclared function 'chroot'; ISO C99 and later do
      not support implicit function declarations [-Wimplicit-function-declaration]
        ret = chroot(str);
              ^
1 error generated.
make: *** [Makefile:1615: ext/standard/dir.lo] Error 1

I actually have no idea, this build procedure works well on Linux. It would be better if someone could provide more clues 🕵️

Environment

Item Value
OS macOS Sonoma 14.4.1
Arch arm64
CC clang (Apple clang version 15.0.0)
CXX clang++
PHP Source Version 8.2.18
mongodb-php-driver Version 1.18.0 (release tarball)
static-php-cli Version main branch

Debug Log

php-src/config.log

Related Issue

https://github.com/crazywhalecc/static-php-cli/issues/281

jmikola commented 5 months ago

Static compilation was broken in 1.17.0 by PHPC-2275. I've opened PHPC-2382 to restore that functionality and will cross-reference a PR once I can successfully reproduce a static build.

As for the build error in ext/standard/dir.c, I assume that relates to -Wimplicit-function-declaration being specified in make EXTRA_CFLAGS="...". That seems entirely unrelated to the MongoDB driver, so you should raise that issue upstream.

crazywhalecc commented 5 months ago

@jmikola Thanks for quick response. I'll test it after it have been fixed.

And for -Wimplicit-function-declaration, it's actually the same with or without it. The error message is the same if EXTRA_CFLAGS is not specified, this error is not reported only when mongodb is not included. (I guess it may be because libmongoc or a library must use C99 features and will not affect each other when compiled independently?)

jmikola commented 5 months ago

I guess it may be because libmongoc or a library must use C99 features and will not affect each other when compiled independently?

libmongoc 1.24+ does require C99 (see: config.m4); however, so does php-src since 8.0.0 (see: https://github.com/php/php-src/commit/b51a99ae358b3295de272bb3c3edb6913eeb0fd4). I don't think the issue is C99 specifically, but rather that implicit function declarations are in error since C99 (see: Implicit function declarations in C Programming. That would explain why changing EXTRA_CFLAGS has no effect.

https://github.com/OpenSCAP/openscap/pull/1626#issuecomment-724098033 is another discussion about chroot() being undefined on macOS and mentions that "chroot() is only defined if _POSIX_SOURCE is undefined.

In scripts/autotools/PlatformFlags.m4, we explicitly enable POSIX features during compilation. That was necessary to address cross-platform compatibility issues with strerror_r() when we upgraded to libmongoc 1.24.3 in PHP driver 1.16.2 (see: PHPC-2270. The relevant PR is https://github.com/mongodb/mongo-php-driver/pull/1458 if you'd like some additional context.

While we never explicitly define _POSIX_SOURCE, we do define some other constants that may be influencing the header declaration of chroot(). I'm not in a good position to investigate that further, given that I don't have access to macOS. The unistd.h linked from https://github.com/OpenSCAP/openscap/pull/1626#issuecomment-724098033 doesn't refer to any of the constants we're defining in PlatformFlags.m4, but that may not be enough to rule out any connection.

In php-src, the chroot() definition is conditional on HAVE_CHROOT (see: ext/standard/dir.c). That detection likely happens before any of the PHP driver's M4 scripts are executed. In addition to HAVE_CHROOT, the conditional also depends on ENABLE_CHROOT_FUNC. Looking at ext/standard/config.m4, it appears to be inferred based on the SAPI (e.g. enabled for CLI, disabled for web servers). I'm not sure if PHP has a means to toggle that during configure time.

You could also look into patching PlatformFlags.m4 to call AC_DEFINE(ENABLE_CHROOT_FUNC, 0) from the macOS conditional and see if that impacts the generation of php_config.h during the PHP build process. I have reservations about making such a change directly in this library, though.

jmikola commented 5 months ago

Per feature_test_macros(7), our definition of _XOPEN_SOURCE=700 results in _POSIX_SOURCE=1 and _POSIX_C_SOURCE=200809L to be defined. Consulting a newer version of unistd.h, we can see how chroot() is left undeclared:

/* Begin XSI */
/* Removed in Issue 6 */
#if !defined(_POSIX_C_SOURCE) || _POSIX_C_SOURCE < 200112L
#if !defined(_POSIX_C_SOURCE)
__deprecated __WATCHOS_PROHIBITED __TVOS_PROHIBITED
#endif
void    *brk(const void *);
int  chroot(const char *) __POSIX_C_DEPRECATED(199506L);
#endif

I expect there may also be another side effect going unnoticed here, since php-src also uses strerror_r() in Zend/zend.c. While that function is always defined, I expect php-src ends up thinking it has a different signature at configuration time than build time.

Looking more closely at PlatformFlags.m4, I think there is room for improvement in how we assign CPPFLAGS. Ideally, any modifications to build flags should be self-contained to the mongodb extension and not influence php-src or other extensions during a static build. I'll look into that further for PHPC-2382.

If you're willing to test my idea, I'd be interested to see the impact of the following modification. In config.m4, backup the $CPPFLAGS variable in the if "$PHP_MONGODB_SYSTEM_LIBS" = "no" conditional here, either before or after we initializing PHP_MONGODB_BUNDLED_CFLAGS and restore the variable at the end of that conditional here. Note the two fi tokens, so make sure you do this after the if test "$PHP_MONGODB_CLIENT_SIDE_ENCRYPTION" = "yes" conditional (which isn't going to be executed given your configure options). For example:


   if test "$PHP_MONGODB_SYSTEM_LIBS" = "no"; then
+    PHP_MONGODB_ORIGINAL_CPPFLAGS="$CPPFLAGS"
     PHP_MONGODB_BUNDLED_CFLAGS="$STD_CFLAGS -DBSON_COMPILATION -DMONGOC_COMPILATION"

     ...

     fi

+    CPPFLAGS="$PHP_MONGODB_ORIGINAL_CPPFLAGS"
   fi
crazywhalecc commented 5 months ago

Applied this patch for backup CPPFLAGS, and it works well. Here's my before and after patch php-src config.log:

I can also provide more details if needed.

crazywhalecc commented 5 months ago

Ideally, any modifications to build flags should be self-contained to the mongodb extension and not influence php-src or other extensions during a static build

Yeah. I found snappy, swoole and some other pecl extensions had this similar issue before, example https://github.com/kjdev/php-ext-snappy/issues/24

People usually use dynamic compilation, and this kind of problem is difficult to detect.

jmikola commented 5 months ago

@crazywhalecc: https://github.com/mongodb/mongo-php-driver/pull/1555 has been merged to v1.19 if you'd like to give that one more try.

I reckon we can release 1.19.1 once we sort out some upstream breaks to Windows builds (PHPC-2388). Hopefully early next week.

crazywhalecc commented 5 months ago

@jmikola I haven't tested every detail in depth, but it compiles fine now without any patches. Thank you !

$ buildroot/bin/php --ri mongodb           

mongodb

MongoDB support => enabled
MongoDB extension version => 1.20.0dev
MongoDB extension stability => devel
libbson bundled version => 1.27.0
libmongoc bundled version => 1.27.0
libmongoc SSL => enabled
libmongoc SSL library => Secure Transport
libmongoc crypto => enabled
libmongoc crypto library => Common Crypto
libmongoc crypto system profile => disabled
libmongoc SASL => disabled
libmongoc SRV => enabled
libmongoc compression => enabled
libmongoc compression snappy => disabled
libmongoc compression zlib => enabled
libmongoc compression zstd => disabled
libmongocrypt => disabled

Directive => Local Value => Master Value
mongodb.debug => no value => no value
jmikola commented 4 months ago

1.19.1 was released with the necessary fix.