search

IBM / ibmi-oss-issues

Important resources for anyone interested in open source on IBM i
Creative Commons Zero v1.0 Universal
13 stars 0 forks source link

IBM i 7.5: Installation of Python 3.9 package lxml fails with gcc error sigset_t #27

Closed epgmat closed 8 months ago

epgmat commented 1 year ago

We have installed a new IBM i 7.5 LPAR and have now gcc errors when installing the Python 3.9 package lxml:

python3.9 -m pip install lxml

Collecting lxml
  Downloading lxml-4.9.2.tar.gz (3.7 MB)
     |████████████████████████████████| 3.7 MB 12.5 MB/s
Building wheels for collected packages: lxml
  Building wheel for lxml (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /QOpenSys/pkgs/bin/python3.9 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-19tos2fa/lxml_4750c219dc2e4a43bb4f6d75333fc91a/setup.py'"'"'; __file__='"'"'/tmp/pip-install-19tos2fa/lxml_4750c219dc2e4a43bb4f6d75333fc91a/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-9szs5lmv
       cwd: /tmp/pip-install-19tos2fa/lxml_4750c219dc2e4a43bb4f6d75333fc91a/
  Complete output (185 lines):
  Building lxml version 4.9.2.
...
  running build_ext
  building 'lxml.etree' extension
  creating build/temp.os400-powerpc64-3.9
  creating build/temp.os400-powerpc64-3.9/src
  creating build/temp.os400-powerpc64-3.9/src/lxml
  gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -O2 -g -maix64 -mtune=power7 -mcpu=power7 -maltivec -mvsx -O2 -g -maix64 -mtune=power7 -mcpu=power7 -maltivec -mvsx -fPIC -DCYTHON_CLINE_IN_TRACEBACK=0 -I/QOpenSys/pkgs/include/libxml2 -I/QOpenSys/pkgs/include -Isrc -Isrc/lxml/includes -I/QOpenSys/pkgs/include/python3.9 -c src/lxml/etree.c -o build/temp.os400-powerpc64-3.9/src/lxml/etree.o -w
  In file included from /usr/include/sys/resource.h:57:0,
                   from /QOpenSys/pkgs/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/include-fixed-7.1/sys/wait.h:56,
                   from /QOpenSys/pkgs/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/include-fixed-7.1/stdlib.h:386,
                   from /QOpenSys/pkgs/include/python3.9/Python.h:34,
                   from src/lxml/etree.c:96:
  /usr/include/sys/time.h:108:16: error: redefinition of 'struct sigset_t'
   typedef struct sigset_t {
                  ^~~~~~~~
  In file included from /QOpenSys/pkgs/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/include-fixed-7.1/stdio.h:501:0,
                   from /QOpenSys/pkgs/include/python3.9/Python.h:25,
                   from src/lxml/etree.c:96:
  /QOpenSys/pkgs/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/include-fixed-7.1/sys/types.h:345:16: note: originally defined here
   typedef struct sigset_t {
                  ^~~~~~~~
  In file included from /usr/include/sys/resource.h:57:0,
                   from /QOpenSys/pkgs/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/include-fixed-7.1/sys/wait.h:56,
                   from /QOpenSys/pkgs/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/include-fixed-7.1/stdlib.h:386,
                   from /QOpenSys/pkgs/include/python3.9/Python.h:34,
                   from src/lxml/etree.c:96:
  /usr/include/sys/time.h:124:3: error: conflicting types for 'sigset_t'
   } sigset_t;
     ^~~~~~~~
  In file included from /QOpenSys/pkgs/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/include-fixed-7.1/stdio.h:501:0,
                   from /QOpenSys/pkgs/include/python3.9/Python.h:25,
                   from src/lxml/etree.c:96:
  /QOpenSys/pkgs/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/include-fixed-7.1/sys/types.h:361:3: note: previous declaration of 'sigset_t' was here
   } sigset_t;
     ^~~~~~~~
  Compile failed: command '/QOpenSys/pkgs/bin/gcc' failed with exit code 1
  creating tmp
  cc -I/QOpenSys/pkgs/include/libxml2 -I/QOpenSys/pkgs/include -I/usr/include/libxml2 -c /tmp/xmlXPathIniti9ng6ahk.c -o tmp/xmlXPathIniti9ng6ahk.o
  In file included from /usr/include/sys/resource.h:57:0,
                   from /QOpenSys/pkgs/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/include-fixed-7.1/sys/wait.h:56,
                   from /QOpenSys/pkgs/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/include-fixed-7.1/stdlib.h:386,
                   from /QOpenSys/pkgs/include/libxml2/libxml/SAX.h:16,
                   from /QOpenSys/pkgs/include/libxml2/libxml/globals.h:20,
                   from /QOpenSys/pkgs/include/libxml2/libxml/xmlIO.h:117,
                   from /QOpenSys/pkgs/include/libxml2/libxml/parser.h:811,
                   from /QOpenSys/pkgs/include/libxml2/libxml/xmlerror.h:10,
                   from /QOpenSys/pkgs/include/libxml2/libxml/xpath.h:26,
                   from /tmp/xmlXPathIniti9ng6ahk.c:1:
  /usr/include/sys/time.h:108:16: error: redefinition of 'struct sigset_t'
   typedef struct sigset_t {
                  ^~~~~~~~
  In file included from /QOpenSys/pkgs/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/include-fixed-7.1/stdio.h:501:0,
                   from /QOpenSys/pkgs/include/libxml2/libxml/tree.h:15,
                   from /QOpenSys/pkgs/include/libxml2/libxml/parser.h:16,
                   from /QOpenSys/pkgs/include/libxml2/libxml/xmlerror.h:10,
                   from /QOpenSys/pkgs/include/libxml2/libxml/xpath.h:26,
                   from /tmp/xmlXPathIniti9ng6ahk.c:1:
  /QOpenSys/pkgs/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/include-fixed-7.1/sys/types.h:345:16: note: originally defined here
   typedef struct sigset_t {
                  ^~~~~~~~
  In file included from /usr/include/sys/resource.h:57:0,
                   from /QOpenSys/pkgs/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/include-fixed-7.1/sys/wait.h:56,
                   from /QOpenSys/pkgs/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/include-fixed-7.1/stdlib.h:386,
                   from /QOpenSys/pkgs/include/libxml2/libxml/SAX.h:16,
                   from /QOpenSys/pkgs/include/libxml2/libxml/globals.h:20,
                   from /QOpenSys/pkgs/include/libxml2/libxml/xmlIO.h:117,
                   from /QOpenSys/pkgs/include/libxml2/libxml/parser.h:811,
                   from /QOpenSys/pkgs/include/libxml2/libxml/xmlerror.h:10,
                   from /QOpenSys/pkgs/include/libxml2/libxml/xpath.h:26,
                   from /tmp/xmlXPathIniti9ng6ahk.c:1:
  /usr/include/sys/time.h:124:3: error: conflicting types for 'sigset_t'
   } sigset_t;
     ^~~~~~~~
  In file included from /QOpenSys/pkgs/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/include-fixed-7.1/stdio.h:501:0,
                   from /QOpenSys/pkgs/include/libxml2/libxml/tree.h:15,
                   from /QOpenSys/pkgs/include/libxml2/libxml/parser.h:16,
                   from /QOpenSys/pkgs/include/libxml2/libxml/xmlerror.h:10,
                   from /QOpenSys/pkgs/include/libxml2/libxml/xpath.h:26,
                   from /tmp/xmlXPathIniti9ng6ahk.c:1:
  /QOpenSys/pkgs/lib/gcc/powerpc-ibm-aix6.1.0.0/6.3.0/include-fixed-7.1/sys/types.h:361:3: note: previous declaration of 'sigset_t' was here
   } sigset_t;
     ^~~~~~~~
  *********************************************************************************
  Could not find function xmlCheckVersion in library libxml2. Is libxml2 installed?
  *********************************************************************************
  error: command '/QOpenSys/pkgs/bin/gcc' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for lxml
  Running setup.py clean for lxml
Failed to build lxml

On IBM i 7.4 the installation runs without errors.

The same issue occurred on AIX 7.2 TL 5: https://community.ibm.com/community/user/power/discussion/gcc-struct-sigset-t-conflicts-with-aix-systimeh

It seems that IBM has fixed this header issue in gcc-10 but not for gcc-6. gcc-6 should also be fixed because IBM i 7.5 uses gcc-6 as the default compiler for their RPM builds: https://kadler.io/2021/04/07/mar-oss-updates.html

Thanks, Matthias

kadler commented 1 year ago

Yes, there are some incompatibilities with GCC 6 on IBM i 7.5. We have no plans to fix GCC 6, but GCC 10 will work. You can use CC=gcc-10 CXX=g++-10 python3.9 -m pip ... on IBM i 7.5.

epgmat commented 1 year ago

In that case, shouldn't gcc-10 be the default compiler for IBM i 7.5? Since the current default compiler (SYMLNK /QOpenSys/pkgs/bin/gcc -> gcc-6) does not work correctly and will not be fixed.

epgmat commented 1 year ago

Yes, there are some incompatibilities with GCC 6 on IBM i 7.5. We have no plans to fix GCC 6, but GCC 10 will work. You can use CC=gcc-10 CXX=g++-10 python3.9 -m pip ... on IBM i 7.5.

When I run python3.9 -m pip -v install lxml (adding parameter verbose), I can see that gcc will be called with parameter -pthread: gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -O2 -g -maix64 -mtune=power7 -mcpu=power7 -maltivec -mvsx -O2 -g -maix64 -mtune=power7 -mcpu=power7 -maltivec -mvsx -fPIC -DCYTHON_CLINE_IN_TRACEBACK=0 -I/QOpenSys/pkgs/include/libxml2 -I/QOpenSys/pkgs/include -Isrc -Isrc/lxml/includes -I/QOpenSys/pkgs/include/python3.9 -c src/lxml/etree.c -o build/temp.os400-powerpc64-3.9/src/lxml/etree.o -w

When using gcc-10 via the environment variables it does not be called with parameter -pthread: gcc-10 -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -O2 -g -maix64 -mtune=power7 -mcpu=power7 -maltivec -mvsx -O2 -g -maix64 -mtune=power7 -mcpu=power7 -maltivec -mvsx -fPIC -DCYTHON_CLINE_IN_TRACEBACK=0 -I/QOpenSys/pkgs/include/libxml2 -I/QOpenSys/pkgs/include -Isrc -Isrc/lxml/includes -I/QOpenSys/pkgs/include/python3.9 -c src/lxml/etree.c -o build/temp.os400-powerpc64-3.9/src/lxml/etree.o -w

So should I add the parameter -pthread to the CC variable? Like: CC="gcc-10 -pthread" CXX=g++-10 python3.9 -m pip ... ?

kadler commented 1 year ago

Ah yeah, looks like the compiler in /QOpenSys/pkgs/lib/python3.9/_sysconfigdata__os400_.py is gcc -pthread, so CC='gcc-10 -pthread' would be appropriate and similarly for CXX.

sebastiangebhardt commented 1 year ago

Thanks for this thread. Was able to install bcrypt==3.1.7 with gcc10 on IBM V7R5. When you don't fix gcc6 anymore shoudn't be gcc10 the default for future releases?

kadler commented 1 year ago

If we had per-release packages, then yeah. We do plan to rebuild everything with GCC 10 and make it the default, but I don't know when that will happen.