search

ovalhub / pyicu

The PyICU project repository has moved to https://pyicu.org.
Other
133 stars 49 forks source link

Build fails on Amazon Linux 2 with PyICU 2.4.2 and ICU 50.1.2 #124

Closed rokclimb15 closed 4 years ago

rokclimb15 commented 4 years ago

Attempted install via pip using wheel on x86_64.

Full log at https://gist.github.com/rokclimb15/fad6270383da63ee2541c4cce95d89c9

ovalhub commented 4 years ago

It looks like your version of gcc doesn't like icu's common.h line 423. What version of gcc are you using ? Also, unrelated, why are you using such an old version of icu ?

On Mar 19, 2020, at 07:46, Brian Morton notifications@github.com wrote:

 Attempted install via pip using wheel on x86_64.

Full log at https://gist.github.com/rokclimb15/fad6270383da63ee2541c4cce95d89c9

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

rokclimb15 commented 4 years ago

These are the default packages on Amazon Linux 2, which is a relatively modern distro. Perhaps these bundled dependencies aren't very modern?

-gcc (GCC) 7.3.1 20180712 (Red Hat 7.3.1-6) -libicu-50.1.2-17.amzn2.x86_64 -libicu-devel-50.1.2-17.amzn2.x86_64

ovalhub commented 4 years ago

gcc 7.3.1 isn't so old but icu 50 sure is.

On Mar 19, 2020, at 10:07, Brian Morton notifications@github.com wrote:

 These are the default packages on Amazon Linux 2, which is a relatively modern distro. Perhaps these bundled dependencies aren't very modern?

-gcc (GCC) 7.3.1 20180712 (Red Hat 7.3.1-6) -libicu-50.1.2-17.amzn2.x86_64 -libicu-devel-50.1.2-17.amzn2.x86_64

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

rokclimb15 commented 4 years ago

Does it seem like the issue here is caused by GCC incompatibility or old ICU, or both?

ovalhub commented 4 years ago

It's most likely some gcc flags. And most likely nothing to do with your version of icu.

On Mar 19, 2020, at 10:22, Brian Morton notifications@github.com wrote:

 Does it seem like the issue here is caused by GCC incompatibility or old ICU, or both?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

ovalhub commented 4 years ago

unless I can reproduce this, I can't help you much :-(

On Mar 19, 2020, at 10:22, Brian Morton notifications@github.com wrote:

 Does it seem like the issue here is caused by GCC incompatibility or old ICU, or both?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

rokclimb15 commented 4 years ago

@ovalhub I'm not an expert with GCC, so if you think of anything, please suggest it. I'll do some research and see if I can figure out what's going on. You can likely reproduce in a Docker environment with docker pull amazonlinux. Thanks!

https://hub.docker.com/_/amazonlinux

ovalhub commented 4 years ago

If you help me reproduce this by giving me access to a system setup with this bug, I can help you. Alternatively, if you list all the command-line statements I need to setup this docker image on a local linux system of mine, I can try to reproduce them. Assume I know nothing about docker and assume I can install any kind of Linux distro as a VM on a local system of mine. Thanks !

On Thu, 19 Mar 2020, Brian Morton wrote:

@ovalhub I'm not an expert with GCC, so if you think of anything, please suggest it. I'll do some research and see if I can figure out what's going on. You can likely reproduce in a Docker environment with docker pull amazonlinux. Thanks!

https://hub.docker.com/_/amazonlinux

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/ovalhub/pyicu/issues/124#issuecomment-601325217

rokclimb15 commented 4 years ago

For security reasons, I can't give you access to a system in my VPC, but this worked for me:

-Go to https://docs.docker.com/install/ -Download and install Docker for your platform -Open a shell or terminal on your system -docker pull amazonlinux -docker run --name amzn-test -it amazonlinux -yum install -y gcc gcc-c++ python3-devel libicu-devel -pip3 install PyICU==2.4.2

Thanks for looking at this!

ovalhub commented 4 years ago

On Thu, 19 Mar 2020, Brian Morton wrote:

For security reasons, I can't give you access to a system in my VPC, but this worked for me:

-Install/start Docker

Please, help me help you. Yes, I can go look up on Google how to do this. It would save me time, if you gave me links and steps on how to Install/start Docker Sorry, for playing stupid but my time is as precious as yours.

In the meantime, I looked at line 423 in common.h and you could try to replace line 407 in common.h: #if defined(_MSC_VER) || defined(PYPY_VERSION) with: #if 1 and see if you get past this problem (that way, you're then in the case of a compiler which doesn't support variadic macros, such as MSVC).

Thanks !

-docker pull amazonlinux -docker run --name amzn-test -it amazonlinux -yum install -y gcc gcc-c++ python3-devel libicu-devel -pip3 install PyICU==2.4.2

Thanks for looking at this!

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/ovalhub/pyicu/issues/124#issuecomment-601349048

ovalhub commented 4 years ago

On Thu, 19 Mar 2020, Andi Vajda wrote:

On Thu, 19 Mar 2020, Brian Morton wrote:

For security reasons, I can't give you access to a system in my VPC, but this worked for me:

-Install/start Docker

Please, help me help you. Yes, I can go look up on Google how to do this. It would save me time, if you gave me links and steps on how to Install/start Docker Sorry, for playing stupid but my time is as precious as yours.

In the meantime, I looked at line 423 in common.h and you could try to replace line 407 in common.h: #if defined(_MSC_VER) || defined(PYPY_VERSION) with: #if 1 and see if you get past this problem (that way, you're then in the case of a compiler which doesn't support variadic macros, such as MSVC).

If you're trying the above, you also need to apply the same change line 883 in common.cpp.

Andi..

Thanks !

-docker pull amazonlinux -docker run --name amzn-test -it amazonlinux -yum install -y gcc gcc-c++ python3-devel libicu-devel -pip3 install PyICU==2.4.2

Thanks for looking at this!

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/ovalhub/pyicu/issues/124#issuecomment-601349048

rokclimb15 commented 4 years ago

Sorry, I left that bit out since it varies a bit per platform. I've edited my original comment to be a bit clearer. I appreciate you taking a look at this!

I tried your suggestion, the build still fails with the same errors.

ovalhub commented 4 years ago

On Thu, 19 Mar 2020, Brian Morton wrote:

Sorry, I left that bit out since it varies a bit per platform. I've edited my original comment to be a bit clearer. I appreciate you taking a look at this!

Go to https://docs.docker.com/install/ Download and install Docker for your platform

Super useful install instructions step by step there ! After installing a 64-bit debian 10 VM, I was able to install and configure docker without a glitch.

Open a shell or terminal on your system docker pull amazonlinux docker run --name amzn-test -it amazonlinux yum install -y gcc gcc-c++ python3-devel libicu-devel pip3 install PyICU==2.4.2

I was able to reproduce the error as reported. Thank you for your help in my getting there !

rokclimb15 commented 4 years ago

Let me know if I can help with a fix. I'm a programmer, but I haven't written c++ in 20 years and don't understand g++/macros at all.

ovalhub commented 4 years ago

After further debugging here are the problems with this setup. Several issues are conspiring to making this platform problematic:

The c++ flags passed to gcc were picked by icu-config: (running 'icu-config --version') Building PyICU 2.4.2 for ICU 50.1.2 (running 'icu-config --cxxflags --cppflags') Adding CFLAGS="-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fno-strict-aliasing -W -Wall -pedantic -Wpointer-arith -Wwrite-strings -Wno-long-long -std=c++11 -D_REENTRANT -I/usr/include" from /usr/bin/icu-config

In other words, you're building PyICU with a version of ICU that was built with -pedantic enabled and that disables support for variadic macros, among other things. You can work this around by adding these extra flags to the 'linux' setting for VER_FLAGS (a kludge for forcing more flags to your gcc command line) in setup.py: -Wno-error -Wno-variadic-macros VER_FLAGS = { 'darwin': ['-DPYICU_VER="%s"' %(VERSION)], 'linux': ['-DPYICU_VER="%s"' %(VERSION), '-Wno-error', '-Wno-variadic-macros'], 'freebsd': ['-DPYICU_VER="%s"' %(VERSION)], 'win32': ['/DPYICU_VER=\"%s\"' %(VERSION)], 'sunos5': ['-DPYICU_VER="%s"' %(VERSION)], 'cygwin': ['-DPYICU_VER="%s"' %(VERSION)], } This will get you past this error, and turn other warnings back into warnings, not errors.

This is the state of trunk at the moment and I just now released it as PyICU 2.4.3 to PyPI.

I really recommend upgrading to the latest ICU, at least version 65.

rokclimb15 commented 4 years ago

Thank you for your help! I might try to build LibICU from source and upgrade the lib while avoiding the -pedantic flag on build.

ovalhub commented 4 years ago

On Thu, 19 Mar 2020, Brian Morton wrote:

Thank you for your help! I might try to build LibICU from source and upgrade the lib while avoiding the -pedantic flag on build.

It's also possible that -pedantic is not set in newer versions of ICU or that the icu-config program doesn't include it (in fact, in newer versions, icu-config is deprecated and pkg-config is used instead).

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/ovalhub/pyicu/issues/124#issuecomment-601427251

ovalhub commented 4 years ago

I made a bunch of fixes in PyICU's trunk today to better support -pedantic. In particular, -pedantic may now be tested by adding PEDANTIC_FLAGS[platform] into _cflags line 204 of setup.py. I tested this on macos (10.14, clang 8.1.0) and linux (debian 10, gcc 8.3.0) with ICU 63.1. I suspect that more errors are likely to happen with different combinations. Problematic warnings can be turned off by adding

pragma diagnostic ignored "-W" or with a -Wno-

addition to PEDANTIC_FLAGS when #pragma is not effective.

On Fri, 20 Mar 2020, Brian Morton wrote:

Closed #124.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/ovalhub/pyicu/issues/124#event-3149709053