Trouble installing PHP "intl" extension

[mbrodala]

I am trying to install the PHP intl extension as required by our code. Inspired by #206, and after a lot of trial and error (push for every change) I now have this:

before_install:
  - ...
  - sudo apt-get update
  - sudo apt-get install --yes php-pear libicu-dev
  - yes '' | pecl upgrade intl
  - echo "extension=intl.so" >> $HOME/.phpenv/versions/$(phpenv version-name)/etc/php.ini

However, this considerably slows down our builds since we need to 1) update the APT package index and 2) compile a PHP extension.

Is this really the only option to install PHP extensions like intl in the Shippable build environment?

Would you consider adding this extension to shippableimages/ubuntu1404_php?

(Successful build: 565720a4f598e60c009ee528)

[mbrodala]

I have now switched to the following, a lot faster but more more hacky solution:

before_install:
  - ...
  - sudo apt-get update
  - sudo apt-get install --yes php5-intl
  - echo "extension=$(dpkg -L php5-intl | grep intl\.so)" >> $HOME/.phpenv/versions/$(phpenv version-name)/etc/php.ini

This works around the problem that the intl.so cannot be found in the current PHP environment if it was installed system-wide by putting the absolute path into the php.ini.

I'd still like to see a cleaner solution. ;-) The big slowdown now is the apt-get update which is still necessary to avoid 404 errors on package download.

[mbrodala]

Could it be that the "intl" module is now available by default? We are seeing a flood of the following messages on current builds:

PHP Warning: Module 'intl' already loaded in Unknown on line 0 Warning: Module 'intl' already loaded in Unknown on line 0

[mbrodala]

Nope, the extension is still missing (just tried a build without the three lines mentioned above and it failed.).

[manishas]

assigning to @himanshu0503 who is working on updating all images.

[himanshu0503]

@mbrodala : We've recently updated our drydock/u14php:tip image. This image has support for php 5.6 and '7.0', and uses pickle for installing extensions. To install intl package you will need to make these changes to your yml -

build_image: drydock/u14php:tip

before_install:
  - ...
  - currentLocation="$PWD"
  - cd /tmp/pickle
  - printf '\n' | bin/pickle install intl
  - cd "$currentLocation"

Can you kindly use this approach, and let me know if you face any difficulty?

[mbrodala]

@himanshu0503 I don't get to this point since the build fails earlier:

phpenv global $SHIPPABLE_PHP_VERSION
rbenv: version `5.5' not installed

I went with pushd /tmp/pickle && printf '\n' | bin/pickle install intl && popd BTW.

What's the reason for the switch to /tmp/pickle? Is pickle not yet installed globally on the image?

[manishas]

@mbrodala we installed only php 5.6 and 7 because earlier versions are deprecated across several other tools.

For example, the latest version of phpunit - 5.1 - only supports php 5.6 and 7. We also had some issues with php-build removing support for pear and pickle not supporting php 5.3.

We'll create a set of images with php 5.3-5.6 and push it to docker hub. these will be tagged with 2015q4, while the new images will be tagged with 2016q1 and will only have versions 5.6-7.

We're building the images and will update this issue when done.

[mbrodala]

Small update: by droping the last line we can at least get rid of the PHP warning. Apparently the extension loading is already part of the default PHP configuration.

Strike that. This did only work due to image caching.

[mbrodala]

@manishas Do you have some news here, especially wrt the new images?

[manishas]

@mbrodala the images have been updated with php 5.5 now. can you try again and let me know if it works?

[mbrodala]

@manishas The before_install step now took 61.66 seconds compared to 36.83 seconds with our current routine. Also an error occurred at the install step:

phpenv config-rm xdebug.ini
rbenv: no such command `config-rm'

(Example build)

[mbrodala]

@manishas After switching from PHP 5.5 to PHP 5.6 we now encounter a different issue with our current setup:

PHP Warning:  PHP Startup: intl: Unable to initialize module
Module compiled with module API=20121212
PHP    compiled with module API=20131226
These options need to match

Any idea how to fix this?

[mbrodala]

@manishas I'm also seeing the phpenv/rbenv error in my current test branch for the new YML format. (last run) Just to check a few things, I've also added phpenv --version and phpenv help:

phpenv --version
rbenv 1.0.0-16-gd6ab394

phpenv help
INFO:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): api.shippable.com
/home/shippable/cexec/dist/main/requests/packages/urllib3/util/ssl_.py:120: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
Usage: rbenv <command> [<args>]

Some useful rbenv commands are:
/root/.phpenv/libexec/rbenv-help: line 85: type: write error: Broken pipe
   commands    List all available rbenv commands
   local       Set or show the local application-specific Ruby version
   global      Set or show the global Ruby version
   shell       Set or show the shell-specific Ruby version
/root/.phpenv/libexec/rbenv-help: line 85: type: write error: Broken pipe
   rehash      Rehash rbenv shims (run this after installing executables)
   version     Show the current Ruby version and its origin
   versions    List all Ruby versions available to rbenv
   which       Display the full path to an executable
   whence      List all Ruby versions that contain the given executable

See `rbenv help <command>' for information on a specific command.
For full documentation, see: https://github.com/rbenv/rbenv#readme

phpenv config-rm xdebug.ini
rbenv: no such command `config-rm'

This looks rather odd from my POV.

[himanshu0503]

@mbrodala : We've looked into phpenv config-rm xdebug.ini issue. Our earlier images were using rbenv 0.4.0 which had support for config-add, config-rm commands, these commands have been removed in rbenv 1.0.0-16-gd6ab394. Here all all the commands which rbenv 1.0.0-16-gd6ab394 supports.

commands
completions
exec
global
help
hooks
init
install
local
prefix
rehash
root
shell
shims
uninstall
update
version
version-file
version-file-read
version-file-write
version-name
version-origin
versions
whence
which

[mbrodala]

@himanshu0503 That explains the issue at hand, thanks. Is there a proper way to fix this? I can execute the rm / cp manually of course but I think there is a cleaner solution.

[mbrodala]

Are there some updates how to install PHP extensions with the new infrastructure/images? (Pickle, ...). The approach with APT doesn't work with PHP 5.6 as mentioned above.

[himanshu0503]

@mbrodala : Looked through the implementation of config-rm and it was removing xdebug.ini file using rm Link.

To install extensions using pickle you can add the command /tmp/pickle/bin/pickle install EXTENSION_NAME.

[mbrodala]

@himanshu0503 This doesn't work that well:

/tmp/pickle/bin/pickle install intl
  /root/8de2fa01-270f-4ac9-afea-a558578f939d.sh: line 58: /tmp/pickle/bin/pickle: No such file or directory

(I'd also expect pickle to be installed in e.g. /usr/bin to have it in the PATH,)

[mbrodala]

OK, I tried this on the shippableimages/ubuntu1404_php image. It looks better on the drydock/u14php image:

/tmp/pickle/bin/pickle install intl
  - Installing intl (latest-stable)
    Downloading: ...

+-----------------------------------+--------+
| Package name                      | intl   |
| Package version (current release) | 3.0.0  |
| Package status                    | stable |
+-----------------------------------+--------+

Now I only need to figure out how to load this installed extension (read: add to php.ini).

[mbrodala]

@himanshu0503 Do you have a hint for the last point?

[mbrodala]

I am also having a hard time getting verbose output or the build log from Pickle to find out e.g. where the intl.so has been stored.

[himanshu0503]

@mbrodala : Can you try using the command echo "extension=intl.so" >> $HOME/.phpenv/versions/$(phpenv version-name)/etc/php.ini to load intl extension.

We've been trying building php projects using intl on our end. Here is the yml for your reference.

Edit: You can use the command rm $HOME/.phpenv/versions/$(phpenv version-name)/etc/conf.d/xdebug.ini to remove the xdebug.ini file

[mbrodala]

@himanshu0503 Sometimes it can be that simple. I was able to enable "intl" this way, thanks a lot! Asking once again: will Pickle be promoted to a regular system command in the near future or will it stay in /tmp for the time being?

And I guess I'll go with the suggested route to manually remove the xdebug.ini.

[himanshu0503]

@mbrodala : Sure, we've opened an internal issue to fix the pickle path. Will update you once the changes are made.

[mbrodala]

@himanshu0503 Let me know if there are some updates.

I'll close this ticket since there is now a good way to install the extension at build time. If it was already part of the Shippable Docker image, that'd be neat though. ;-)

[tdutrion]

Could you explain how to install the extension at build time? I need intl on PHP7 and can't get it to work :(

[himanshu0503]

@mbrodala : We've created the symbolic link for pickle, now you can install extensions by pickle install $ext.

[mbrodala]

@tdutrion Simply add the following line in your ci (or old script) section:

- /tmp/pickle/bin/pickle install intl && echo "extension=intl.so" >> $HOME/.phpenv/versions/$(phpenv version-name)/etc/php.ini

@himanshu0503 Thanks, I'l give it a try. :-)

[mbrodala]

@himanshu0503 That didn't work out:

pickle install intl && echo "extension=intl.so" >> $HOME/.phpenv/versions/$(phpenv version-name)/etc/php.ini
  /root/f97c8bf3-e73e-4ef9-a06f-d6dcb9faed4a.sh: line 58: pickle: command not found

Am I missing something?

[himanshu0503]

@mbrodala : Do you have a pre_ci_boot section in your yml?

[mbrodala]

@himanshu0503 Yes, it's the same for all of our projects:

  pre_ci_boot:
    image_name: drydock/u14php
    image_tag: tip

[himanshu0503]

@mbrodala : Got the problem, we've only updated the prod tag with the changes so your yml will look like

  pre_ci_boot:
    image_name: drydock/u14php
    image_tag: prod
    pull: true

If you don't specify the pull: true flag, we will use the image present on the system, and this can lead to your builds running on a stale image.

Alternatively, ill suggest you to remove the pre_ci_boot section for drydock images as by default we use the drydock/u14$lang:prod image for building your projects.

[tdutrion]

Can't get it to work either... I have no pre_ci_boot section, and trying to install intl, redis and gettext.

Here is a sample of my config

language: php

services:
  - postgres
  - redis

php:
  - 7.0

build:
  ci:
    - /tmp/pickle/bin/pickle install intl && echo "extension=intl.so" >> $HOME/.phpenv/versions/$(phpenv version-name)/etc/php.ini
    - /tmp/pickle/bin/pickle install redis && echo "extension=redis.so" >> $HOME/.phpenv/versions/$(phpenv version-name)/etc/php.ini

For now I mainly have a compilation error for intl and another error for redis...

/tmp/pickle/bin/pickle install intl && echo "extension=intl.so" >> $HOME/.phpenv/versions/$(phpenv version-name)/etc/php.ini
  - Installing intl (latest-stable)
    Downloading: Connecting...    Downloading: 0%               Downloading: 10%    Downloading: 20%    Downloading: 30%    Downloading: 40%    Downloading: 50%    Downloading: 100%    Downloading: 100%

+-----------------------------------+--------+
| Package name                      | intl   |
| Package version (current release) | 3.0.0  |
| Package status                    | stable |
+-----------------------------------+--------+
The following error(s) happened: make failed
1: phpize
2: Configuring for:
2: PHP Api Version:         20151012
2: Zend Module Api No:      20151012
2: Zend Extension Api No:   320151012
1: /tmp/intl/intl-3.0.0/configure --enable-intl=shared 
2: checking for grep that handles long lines and -e... /bin/grep
2: checking for egrep... /bin/grep -E
2: checking for a sed that does not truncate output... /bin/sed
2: checking for cc... cc
2: checking whether the C compiler works... yes
2: checking for C compiler default output file name... a.out
2: checking for suffix of executables...
2: checking whether we are cross compiling... no
2: checking for suffix of object files... o
2: checking whether we are using the GNU C compiler... yes
2: checking whether cc accepts -g... yes
2: checking for cc option to accept ISO C89... none needed
2: checking how to run the C preprocessor... cc -E
2: checking for icc... no
2: checking for suncc... no
2: checking whether cc understands -c and -o together... yes
2: checking for system library directory... lib
2: checking if compiler supports -R... no
2: checking if compiler supports -Wl,-rpath,... yes
2: checking build system type... x86_64-unknown-linux-gnu
2: checking host system type... x86_64-unknown-linux-gnu
2: checking target system type... x86_64-unknown-linux-gnu
2: checking for PHP prefix... /root/.phpenv/versions/7.0
2: checking for PHP includes... -I/root/.phpenv/versions/7.0/include/php -I/root/.phpenv/versions/7.0/include/php/main -I/root/.phpenv/versions/7.0/include/php/TSRM -I/root/.phpenv/versions/7.0/include/php/Zend -I/root/.phpenv/versions/7.0/include/php/ext -I/root/.phpenv/versions/7.0/include/php/ext/date/lib
2: checking for PHP extension directory... /root/.phpenv/versions/7.0/lib/php/extensions/no-debug-non-zts-20151012
2: checking for PHP installed headers prefix... /root/.phpenv/versions/7.0/include/php
2: checking if debug is enabled... no
2: checking if zts is enabled... no
2: checking for re2c... no
2: configure: WARNING: You will need re2c 0.13.4 or later if you want to regenerate PHP parsers.
2: checking for gawk... gawk
2: checking whether to enable internationalization support... yes, shared
2: checking for icu-config... /usr/bin/icu-config
2: checking for location of ICU headers and libraries... /usr
2: checking for ICU 4.0 or greater... found 52.1
2: checking for g++... g++
2: checking whether we are using the GNU C++ compiler... yes
2: checking whether g++ accepts -g... yes
2: checking how to run the C++ preprocessor... g++ -E
2: checking for ld used by cc... /usr/bin/ld
2: checking if the linker (/usr/bin/ld) is GNU ld... yes
2: checking for /usr/bin/ld option to reload object files... -r
2: checking for BSD-compatible nm... /usr/bin/nm -B
2: checking whether ln -s works... yes
2: checking how to recognize dependent libraries... pass_all
2: checking for ANSI C header files... yes
2: checking for sys/types.h... yes
2: checking for sys/stat.h... yes
2: checking for stdlib.h... yes
2: checking for string.h... yes
2: checking for memory.h... yes
2: checking for strings.h... yes
2: checking for inttypes.h... yes
2: checking for stdint.h... yes
2: checking for unistd.h... yes
2: checking dlfcn.h usability... yes
2: checking dlfcn.h presence... yes
2: checking for dlfcn.h... yes
2: checking how to run the C++ preprocessor... g++ -E
2: checking the maximum length of command line arguments... 1572864
2: checking command to parse /usr/bin/nm -B output from cc object... ok
2: checking for objdir... .libs
2: checking for ar... ar
2: checking for ranlib... ranlib
2: checking for strip... strip
2: checking if cc supports -fno-rtti -fno-exceptions... no
2: checking for cc option to produce PIC... -fPIC
2: checking if cc PIC flag -fPIC works... yes
2: checking if cc static flag -static works... yes
2: checking if cc supports -c -o file.o... yes
2: checking whether the cc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
2: checking whether -lc should be explicitly linked in... no
2: checking dynamic linker characteristics... GNU/Linux ld.so
2: checking how to hardcode library paths into programs... immediate
2: checking whether stripping libraries is possible... yes
2: checking if libtool supports shared libraries... yes
2: checking whether to build shared libraries... yes
2: checking whether to build static libraries... no
2: 
2: creating libtool
2: appending configuration tag "CXX" to libtool
2: checking for ld used by g++... /usr/bin/ld -m elf_x86_64
2: checking if the linker (/usr/bin/ld -m elf_x86_64) is GNU ld... yes
2: checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
2: checking for g++ option to produce PIC... -fPIC
2: checking if g++ PIC flag -fPIC works... yes
2: checking if g++ static flag -static works... yes
2: checking if g++ supports -c -o file.o... yes
2: checking whether the g++ linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
2: checking dynamic linker characteristics... GNU/Linux ld.so
2: (cached) (cached) checking how to hardcode library paths into programs... immediate
2: configure: creating ./config.status
2: config.status: creating config.h
1: make
2: /bin/bash /tmp/pickle-bbe1627712054a1e2aed0525aed282d333bfc5c4/libtool --mode=compile cc -I/usr/include  -Wno-write-strings -I. -I/tmp/intl/intl-3.0.0 -DPHP_ATOM_INC -I/tmp/pickle-bbe1627712054a1e2aed0525aed282d333bfc5c4/include -I/tmp/pickle-bbe1627712054a1e2aed0525aed282d333bfc5c4/main -I/tmp/intl/intl-3.0.0 -I/root/.phpenv/versions/7.0/include/php -I/root/.phpenv/versions/7.0/include/php/main -I/root/.phpenv/versions/7.0/include/php/TSRM -I/root/.phpenv/versions/7.0/include/php/Zend -I/root/.phpenv/versions/7.0/include/php/ext -I/root/.phpenv/versions/7.0/include/php/ext/date/lib  -DHAVE_CONFIG_H  -g -O2   -c /tmp/intl/intl-3.0.0/php_intl.c -o php_intl.lo
2: mkdir .libs
2:  cc -I/usr/include -Wno-write-strings -I. -I/tmp/intl/intl-3.0.0 -DPHP_ATOM_INC -I/tmp/pickle-bbe1627712054a1e2aed0525aed282d333bfc5c4/include -I/tmp/pickle-bbe1627712054a1e2aed0525aed282d333bfc5c4/main -I/tmp/intl/intl-3.0.0 -I/root/.phpenv/versions/7.0/include/php -I/root/.phpenv/versions/7.0/include/php/main -I/root/.phpenv/versions/7.0/include/php/TSRM -I/root/.phpenv/versions/7.0/include/php/Zend -I/root/.phpenv/versions/7.0/include/php/ext -I/root/.phpenv/versions/7.0/include/php/ext/date/lib -DHAVE_CONFIG_H -g -O2 -c /tmp/intl/intl-3.0.0/php_intl.c  -fPIC -DPIC -o .libs/php_intl.o
2: In file included from /tmp/intl/intl-3.0.0/php_intl.h:34:0,
2:                  from /tmp/intl/intl-3.0.0/php_intl.c:25:
2: /tmp/intl/intl-3.0.0/intl_error.h:24:40: fatal error: ext/standard/php_smart_str.h: No such file or directory

            2:                                         ^
2: compilation terminated.
2: make: *** [php_intl.lo] Error 1
  /tmp/pickle/bin/pickle install redis && echo "extension=redis.so" >> $HOME/.phpenv/versions/$(phpenv version-name)/etc/php.ini
PHP Warning:  PHP Startup: Unable to load dynamic library '/root/.phpenv/versions/7.0/lib/php/extensions/no-debug-non-zts-20151012/intl.so' - /root/.phpenv/versions/7.0/lib/php/extensions/no-debug-non-zts-20151012/intl.so: cannot open shared object file: No such file or directory in Unknown on line 0

Warning: PHP Startup: Unable to load dynamic library '/root/.phpenv/versions/7.0/lib/php/extensions/no-debug-non-zts-20151012/intl.so' - /root/.phpenv/versions/7.0/lib/php/extensions/no-debug-non-zts-20151012/intl.so: cannot open shared object file: No such file or directory in Unknown on line 0
PHP Warning:  PHP Startup: Unable to load dynamic library '/root/.phpenv/versions/7.0/lib/php/extensions/no-debug-non-zts-20151012/intl.so' - /root/.phpenv/versions/7.0/lib/php/extensions/no-debug-non-zts-20151012/intl.so: cannot open shared object file: No such file or directory in Unknown on line 0

  [Exception]                                   
  Could not determine info from the PHP binary  

install [--no-convert] [--defaults] [--source] [--with-configure-options WITH-CONFIGURE-OPTIONS] [--save-logs SAVE-LOGS] [--dry-run] [--php PHP] [--ini INI] [--tmp-dir TMP-DIR] [--] []

[mbrodala]

@tdutrion That sounds like an issue with PHP 7; I'm using 5.6 here.

[mbrodala]

@himanshu0503 Will that slow down builds? It sounds like this will cause more image pulls. And so far I thought that tip is always the latest version.

[himanshu0503]

@mbrodala : tip is the latest version of the images, once the testing is done we update the tip tag to prod. In this case we had to patch the prod tag with the changes.

Regarding builds slowdown: We'll update our AMI's with the latest images in the coming week, so image pulls wont be happening there after.

[mbrodala]

@himanshu0503 Thanks, works for me.

[mbrodala]

@tdutrion Can you open a separate ticket for the issue you see?

[himanshu0503]

@tdutrion : This is a php 7.0 specific issue. We've faced the same issues while installing packages on php 7.0 using pickle and due to this we've not pre-installed redis on our php images for php 7.0. Versions 5.4, 5.5 and 5.6 have redis by default.

[mbrodala]

@himanshu0503 In light of #2300: are there directories which could be cached between builds to speed up extension installations by Pickle?

[mbrodala]

Apparently it's $HOME/.phpenv/versions/$(phpenv version-name)/lib/php/extensions/no-debug-non-zts-20131226/ which cannot be added to cache_dir_list this way. Caching/restoring $HOME/.phpenv/ takes ages thus is not an option.

And Pickle doesn't check if a PHP extension already exists anyways, thus no point in going any further here.

[mbrodala]

Digging further into this we may be able to check ourselves if intl is already present, e.g. via php -m | grep intl.

@himanshu0503, @manishas Any idea how we could add the mentioned extension directory to the cache_dir_list?

[mbrodala]

Continuing in #2553

[tapahbl4]

PHP Warning: PHP Startup: Unable to load dynamic library '/usr/lib/php/modules/intl.so' - /usr/lib/php/modules/intl.so: undefined symbol: _ZTIN6icu_5617StringEnumerationE in Unknown on line 0 What should I do?

[ambarish2012]

@mbrodala intl is available in the v5.4.1 image.

[mbrodala]

Seems to work, my script to install missing extensions via Pickle is not triggered and Composer is satisfied too. Thanks a lot. :-)

[mbrodala]

We just noticed that the Shippable images v5.4.1 drop support for PHP 5.5 which unfortunately is still required in a few of our projects. Thus we had to revert to Stable again.