search

SAP / PyRFC

Asynchronous, non-blocking SAP NW RFC SDK bindings for Python
http://sap.github.io/PyRFC
Apache License 2.0
500 stars 132 forks source link

Error installing on RHEL v8.6, with NW RFC SDK v7.50 PL12 #339

Closed joanba68 closed 11 months ago

joanba68 commented 11 months ago

We've installed NW RFC SDK v7.50 PL 12 in our server running RHEL v8.6. As root we set en environment:

export SAPNWRFC_HOME=/tmp/sap_installers/nwrfcsdk

and run the installation:

pip3 install pyrfc

but fails immediately:

[root@useazsbxss199 sap_installers]# pip3 install pyrfc
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting pyrfc
  Using cached https://files.pythonhosted.org/packages/c2/3f/282b01c8a76da8610b76209ffd825ce999797b7aef2974acbb55ad43ab1d/pyrfc-2.8.2.tar.gz
    Complete output from command python setup.py egg_info:
    750 Patch Level 12 Hotfix 1
    pyrfc version 2.8.2
    Compiling src/pyrfc/_cyrfc.pyx because it changed.
    [1/1] Cythonizing src/pyrfc/_cyrfc.pyx
    /usr/local/lib64/python3.6/site-packages/Cython/Compiler/Main.py:381: FutureWarning: Cython directive 'language_level' not set, using '3str' for now (Py3). This has changed from earlier releases! File: /tmp/pip-build-tx0ez26k/pyrfc/src/pyrfc/_cyrfc.pyx
      tree = Parsing.p_module(s, pxd, full_module_name)
    performance hint: src/pyrfc/_cyrfc.pyx:1571:5: Exception check on 'metadataLookup' will always require the GIL to be acquired. Possible solutions:
        1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.

    performance hint: src/pyrfc/_cyrfc.pyx:1604:5: Exception check on 'genericHandler' will always require the GIL to be acquired. Possible solutions:
        1. Declare the function as 'noexcept' if you control the definition and you're sure you don't want the function to raise exceptions.

    Error compiling Cython file:
    ------------------------------------------------------------
    ...
        def install_bgrfc_handlers(self, sysId):
            ucSysId = fillString(sysId) if sysId is not None else NULL
            cdef RFC_ERROR_INFO errorInfo
            cdef RFC_RC rc = RfcInstallBgRfcHandlers(
                                ucSysId,
                                Server.__onCheckFunction,
                                      ^
    ------------------------------------------------------------

    src/pyrfc/_cyrfc.pyx:1898:34: Cannot assign type 'RFC_RC (RFC_CONNECTION_HANDLE, const RFC_UNIT_IDENTIFIER *) except * nogil' to 'RFC_ON_CHECK_UNIT'

    Error compiling Cython file:
    ------------------------------------------------------------
    ...
            ucSysId = fillString(sysId) if sysId is not None else NULL
            cdef RFC_ERROR_INFO errorInfo
            cdef RFC_RC rc = RfcInstallBgRfcHandlers(
                                ucSysId,
                                Server.__onCheckFunction,
                                Server.__onCommitFunction,
                                      ^
    ------------------------------------------------------------

    src/pyrfc/_cyrfc.pyx:1899:34: Cannot assign type 'RFC_RC (RFC_CONNECTION_HANDLE, const RFC_UNIT_IDENTIFIER *) except * nogil' to 'RFC_ON_COMMIT_UNIT'

    Error compiling Cython file:
    ------------------------------------------------------------
    ...
            cdef RFC_ERROR_INFO errorInfo
            cdef RFC_RC rc = RfcInstallBgRfcHandlers(
                                ucSysId,
                                Server.__onCheckFunction,
                                Server.__onCommitFunction,
                                Server.__onRollbackFunction,
                                      ^
    ------------------------------------------------------------

    src/pyrfc/_cyrfc.pyx:1900:34: Cannot assign type 'RFC_RC (RFC_CONNECTION_HANDLE, const RFC_UNIT_IDENTIFIER *) except * nogil' to 'RFC_ON_ROLLBACK_UNIT'

    Error compiling Cython file:
    ------------------------------------------------------------
    ...
            cdef RFC_RC rc = RfcInstallBgRfcHandlers(
                                ucSysId,
                                Server.__onCheckFunction,
                                Server.__onCommitFunction,
                                Server.__onRollbackFunction,
                                Server.__onConfirmFunction,
                                      ^
    ------------------------------------------------------------

    src/pyrfc/_cyrfc.pyx:1901:34: Cannot assign type 'RFC_RC (RFC_CONNECTION_HANDLE, const RFC_UNIT_IDENTIFIER *) except * nogil' to 'RFC_ON_CONFIRM_UNIT'

    Error compiling Cython file:
    ------------------------------------------------------------
    ...
                                ucSysId,
                                Server.__onCheckFunction,
                                Server.__onCommitFunction,
                                Server.__onRollbackFunction,
                                Server.__onConfirmFunction,
                                Server.__onGetStateFunction,
                                      ^
    ------------------------------------------------------------

    src/pyrfc/_cyrfc.pyx:1902:34: Cannot assign type 'RFC_RC (RFC_CONNECTION_HANDLE, const RFC_UNIT_IDENTIFIER *, RFC_UNIT_STATE *) except * nogil' to 'RFC_ON_GET_UNIT_STATE'

    Error compiling Cython file:
    ------------------------------------------------------------
    ...
            :raises: :exc:`~pyrfc.RFCError` or a subclass
                     thereof if the server processing fails.
            """
            cdef RFC_ERROR_INFO errorInfo

            cdef RFC_RC rc = RfcInstallGenericServerFunction(genericHandler, metadataLookup, &errorInfo)
                                                             ^
    ------------------------------------------------------------

    src/pyrfc/_cyrfc.pyx:1956:57: Cannot assign type 'RFC_RC (RFC_CONNECTION_HANDLE, RFC_FUNCTION_HANDLE, RFC_ERROR_INFO *) except * nogil' to 'RFC_SERVER_FUNCTION'

    Error compiling Cython file:
    ------------------------------------------------------------
    ...
            :raises: :exc:`~pyrfc.RFCError` or a subclass
                     thereof if the server processing fails.
            """
            cdef RFC_ERROR_INFO errorInfo

            cdef RFC_RC rc = RfcInstallGenericServerFunction(genericHandler, metadataLookup, &errorInfo)
                                                                             ^
    ------------------------------------------------------------

    src/pyrfc/_cyrfc.pyx:1956:73: Cannot assign type 'RFC_RC (const SAP_UC *, RFC_ATTRIBUTES, RFC_FUNCTION_DESC_HANDLE *) except * nogil' to 'RFC_FUNC_DESC_CALLBACK'
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-tx0ez26k/pyrfc/setup.py", line 236, in <module>
        if BUILD_CYTHON
      File "/usr/local/lib64/python3.6/site-packages/Cython/Build/Dependencies.py", line 1154, in cythonize
        cythonize_one(*args)
      File "/usr/local/lib64/python3.6/site-packages/Cython/Build/Dependencies.py", line 1321, in cythonize_one
        raise CompileError(None, pyx_file)
    Cython.Compiler.Errors.CompileError: src/pyrfc/_cyrfc.pyx

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-tx0ez26k/pyrfc/

Any idea ?

bsrdjan commented 11 months ago

It fails because pyrfc version 2.8.2 cached source is fetched for some reason and that old version can't be built with newer Cython

 Using cached https://files.pythonhosted.org/packages/c2/3f/282b01c8a76da8610b76209ffd825ce999797b7aef2974acbb55ad43ab1d/pyrfc-2.8.2.tar.gz

Which python version is used to run the pyrfc install command? You can try to fix pyrfc version 3.3 in pip install but it should work without it.

Can you also install pyrfc in user space, not root?

joanba68 commented 11 months ago

Hi,

I'm able to fix the installation but still not able to use pyrfc. I'm not sure why we get the files from old version as I've no access to check it. What I've done to install is to simply download the .zip package from this site (PyRFC-main.zip) and unzip in server. From here, just to set the envvar SAPNWRFC_HOME and install:

[root@useazsbxss199 sap_installers]# cd PyRFC-main/
[root@useazsbxss199 PyRFC-main]# pip3 install .
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Processing /tmp/sap_installers/PyRFC-main
Installing collected packages: pyrfc
  Running setup.py install for pyrfc ... done
Successfully installed pyrfc-0.0.0
[root@useazsbxss199 PyRFC-main]#

still done as root, we will change it later when fix errors. Why it's showing v0.0.0 ?

Now error is different:

[cofcoadm@useazsbxss199 ~]$ python3
Python 3.6.8 (default, Jun 14 2022, 12:54:58)
[GCC 8.5.0 20210514 (Red Hat 8.5.0-10)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pyrfc import *
>>> from pyrfc import Connection
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'Connection'
>>>
bsrdjan commented 11 months ago

Python 3.6 is not supported PyRFC, see: https://pypi.org/project/pyrfc/

Many Linux systems still use older pre-installed system Python and virtual environment installation is therefore recommended, using pyenv for example. It provides supported Python and avoids running applications as root.

I need to investigate why the installation as you described did not fail with unsupported Python.

joanba68 commented 11 months ago

I've uninstalled python v3.6 ( from our OS image ) and installed v3.9.7. Now I've an error during installation again:

[root@useazsbxss199 sap_installers]# pip3 install pyrfc --user
WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.
Collecting pyrfc
  Downloading pyrfc-3.3.tar.gz (352 kB)
     |████████████████████████████████| 352 kB 579 kB/s
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Building wheels for collected packages: pyrfc
  Building wheel for pyrfc (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3.9 /usr/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmpuc_5ewsl
       cwd: /tmp/pip-install-mr87mwzt/pyrfc
  Complete output (17 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-cpython-39
  creating build/lib.linux-x86_64-cpython-39/pyrfc
  copying src/pyrfc/__init__.py -> build/lib.linux-x86_64-cpython-39/pyrfc
  copying src/pyrfc/_exception.py -> build/lib.linux-x86_64-cpython-39/pyrfc
  copying src/pyrfc/_utils.py -> build/lib.linux-x86_64-cpython-39/pyrfc
  running build_ext
  building 'pyrfc._cyrfc' extension
  creating build/temp.linux-x86_64-cpython-39
  creating build/temp.linux-x86_64-cpython-39/src
  creating build/temp.linux-x86_64-cpython-39/src/pyrfc
  gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DNDEBUG -D_LARGEFILE_SOURCE -D_CONSOLE -D_FILE_OFFSET_BITS=64 -DSAPonUNIX -DSAPwithUNICODE -DSAPwithTHREADS -DSAPonLIN -I/usr/include/python3.9 -c src/pyrfc/_cyrfc.cpp -o build/temp.linux-x86_64-cpython-39/src/pyrfc/_cyrfc.o -Wall -O2 -fexceptions -funsigned-char -fno-strict-aliasing -Wall -Wno-uninitialized -Wno-deprecated-declarations -Wno-unused-function -Wcast-align -fPIC -pthread -minline-all-stringops -I/tmp/sap_installers/nwrfcsdk/include
  gcc: error trying to exec 'cc1plus': execvp: No such file or directory
  error: command '/bin/gcc' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for pyrfc
Failed to build pyrfc
ERROR: Could not build wheels for pyrfc which use PEP 517 and cannot be installed directly
[root@useazsbxss199 sap_installers]#
joanba68 commented 11 months ago

We're advancing and getting new errors, previous one was probably because an old pip3 version. Upgraded with:

pip3 install --upgrade pip

Now error has changed:

[root@useazsbxss199 sap_installers]# pip3 install pyrfc --user
Collecting pyrfc
  Using cached pyrfc-3.3.tar.gz (352 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Building wheels for collected packages: pyrfc
  Building wheel for pyrfc (pyproject.toml) ... error
  error: subprocess-exited-with-error

  × Building wheel for pyrfc (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [17 lines of output]
      running bdist_wheel
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-39
      creating build/lib.linux-x86_64-cpython-39/pyrfc
      copying src/pyrfc/__init__.py -> build/lib.linux-x86_64-cpython-39/pyrfc
      copying src/pyrfc/_exception.py -> build/lib.linux-x86_64-cpython-39/pyrfc
      copying src/pyrfc/_utils.py -> build/lib.linux-x86_64-cpython-39/pyrfc
      running build_ext
      building 'pyrfc._cyrfc' extension
      creating build/temp.linux-x86_64-cpython-39
      creating build/temp.linux-x86_64-cpython-39/src
      creating build/temp.linux-x86_64-cpython-39/src/pyrfc
      gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -Wp,-D_GLIBCXX_ASSERTIONS -fexceptions -fstack-protector-strong -grecord-gcc-switches -m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-protection -D_GNU_SOURCE -fPIC -fwrapv -fPIC -DNDEBUG -D_LARGEFILE_SOURCE -D_CONSOLE -D_FILE_OFFSET_BITS=64 -DSAPonUNIX -DSAPwithUNICODE -DSAPwithTHREADS -DSAPonLIN -I/usr/include/python3.9 -c src/pyrfc/_cyrfc.cpp -o build/temp.linux-x86_64-cpython-39/src/pyrfc/_cyrfc.o -Wall -O2 -fexceptions -funsigned-char -fno-strict-aliasing -Wall -Wno-uninitialized -Wno-deprecated-declarations -Wno-unused-function -Wcast-align -fPIC -pthread -minline-all-stringops -I/tmp/sap_installers/nwrfcsdk/include
      gcc: error trying to exec 'cc1plus': execvp: No such file or directory
      error: command '/bin/gcc' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for pyrfc
Failed to build pyrfc
ERROR: Could not build wheels for pyrfc, which is required to install pyproject.toml-based projects
bsrdjan commented 11 months ago

Something is apparently wrong with C++ development toolchain on your system. Cython transforms the Cython source to C++ and C++ compiler required to build. 

error: command '/bin/gcc' failed with exit code 1

Perhaps this helps: https://access.redhat.com/solutions/1137503 but I have no access.

Here how the toolchain is installed on Ubuntu system, used in PyRFC testing: https://github.com/SAP/fundamental-tools/blob/main/docker/ubuntu-qa.Dockerfile#L22

bsrdjan commented 11 months ago

see also https://stackoverflow.com/questions/11912878/gcc-error-gcc-error-trying-to-exec-cc1-execvp-no-such-file-or-directory

joanba68 commented 11 months ago

Finally got installed pyrfc 3.3.

dnf install gcc-c++
# python v3.9.7 installed
dnf install python39-devel
[root@useazsbxss199 sap_installers]# pip3 install --upgrade pip
(...)
Successfully installed pip-23.2.1

and finally got a successful installation of pyrfc:

[root@useazsbxss199 sap_installers]# pip3 install pyrfc --user
Collecting pyrfc
  Using cached pyrfc-3.3.tar.gz (352 kB)
(...)
Successfully installed pyrfc-3.3

Unfortunately we still not have a working pyrfc:

[root@useazsbxss199 sap_installers]# python
Python 3.9.7 (default, Jun 14 2023, 09:39:13)
[GCC 8.5.0 20210514 (Red Hat 8.5.0-10)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pyrfc import Connection
libsapnwrfc.so: cannot open shared object file: No such file or directory
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: cannot import name 'Connection' from 'pyrfc' (/root/.local/lib/python3.9/site-packages/pyrfc/__init__.py)
>>>

Moving to a user, we've repeated pyrfc installation and finally have it working but adding a new env variable: LD_LIBRARY_PATH

[cofcoadm@useazsbxss199 ~]$ export LD_LIBRARY_PATH=/tmp/sap_installers/nwrfcsdk/lib
[cofcoadm@useazsbxss199 ~]$ python
Python 3.9.7 (default, Jun 14 2023, 09:39:13)
[GCC 8.5.0 20210514 (Red Hat 8.5.0-10)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pyrfc import Connection
>>>

So at this point I understand we have pyrfc properly installed, but still failing in ansible task. But that's subject of another repo.

Thanks !!

bsrdjan commented 11 months ago

Added the note IN README about Linux https://github.com/SAP/PyRFC#linux, to help other users with similar issues