ydk installation issues on some CentOS environments

ghost commented 7 years ago

Need to fix ydk 0.6.0 installation for core so that cmake3 or cmake is used without having to link the binary

ydk installation should not assume shared library for python is installed

[ 50%] Building CXX object CMakeFiles/ydk_.dir/python.cpp.o
/tmp/pip-build-uotkxcx1/ydk/python.cpp:551:198: warning: unused parameter ���port��� [-Wunused-parameter]
             [](ydk::NetconfServiceProvider &nc_provider, const string& address, const string& username, const string& password, void* port, const string& protocol, bool on_demand, bool common_cache) {
                                                                                                                                                                                                      ^
/tmp/pip-build-uotkxcx1/ydk/python.cpp:551:198: warning: unused parameter ���port��� [-Wunused-parameter]
[100%] Linking CXX shared module ../lib.linux-x86_64-3.6/ydk_.so
/usr/bin/ld: /usr/local/lib/python3.6/config-3.6m-x86_64-linux-gnu/libpython3.6m.a(abstract.o): relocation R_X86_64_32S against `_Py_NotImplementedStruct' can not be used when making a shared object; recompile with -fPIC
/usr/local/lib/python3.6/config-3.6m-x86_64-linux-gnu/libpython3.6m.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
gmake[2]: *** [../lib.linux-x86_64-3.6/ydk_.so] Error 1
gmake[1]: *** [CMakeFiles/ydk_.dir/all] Error 2
gmake: *** [all] Error 2
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/tmp/pip-build-uotkxcx1/ydk/setup.py", line 157, in <module>
    zip_safe=False,
  File "/usr/local/lib/python3.6/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/local/lib/python3.6/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/users/skopperl/cafy_env/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
    return orig.install.run(self)
  File "/usr/local/lib/python3.6/distutils/command/install.py", line 545, in run
    self.run_command('build')
  File "/usr/local/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.6/distutils/command/build.py", line 135, in run
    self.run_command(cmd_name)
  File "/usr/local/lib/python3.6/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python3.6/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/tmp/pip-build-uotkxcx1/ydk/setup.py", line 69, in run
    self.build_extension(ext)
  File "/tmp/pip-build-uotkxcx1/ydk/setup.py", line 94, in build_extension
    subprocess.check_call(['cmake', '--build', '.'], cwd=self.build_temp)
  File "/usr/local/lib/python3.6/subprocess.py", line 291, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '--build', '.']' returned non-zero exit status 2.

----------------------------------------

ghost commented 6 years ago

Currently, there is a need to rebuild python library on some CentOS environments with --enable-shared option to compile ydk-py. Could be fixed with #626

ghost commented 6 years ago

Couple of troubleshooting workarounds for CentOS:

You may face the below issues when doing something like pip install ydk on CentOS:

1) python installation does not have shared libraries

You may see something like the below error:

...
[100%] Linking CXX shared module ../lib.linux-x86_64-2.7/ydk_.so
  /usr/bin/ld: /usr/local/lib/libpython2.7.a(abstract.o): relocation R_X86_64_32S against `_Py_NotImplementedStruct' can not be used when making a shared object; recompile with -fPIC
...

To fix this, you need to compile the python installation with the --enable-shared option as follows (steps obtained from google search):

Retrieve the sources from the official website: http://www.python.org/download/

Unpack, compile and install Python (any version, here with Python 2.7):

cd <compilation-directory>
tar jvzf Python-2.7.tar.bz2
cd Python-2.7
./configure --enable-shared [--prefix=/your/custom/installation/path]
make
make test
make install

2) cmake cannot find the correct python install location

    ...
    python version: 3.6m
    -- Found PythonLibs: /usr/local/lib/libpython2.7.a (found version "2.7")
    pybind11 include: /root/user/Python-3.6.2/include/python3.6m;/root/.local/include/python3.6m
   ...

  [100%] Linking CXX shared module ../lib.linux-x86_64-3.6/ydk_.so
    /usr/bin/ld: /usr/local/lib/libpython2.7.a(abstract.o): relocation R_X86_64_32S against `_Py_NotImplementedStruct' can not be used when making a shared object; recompile with -fPIC
    /usr/local/lib/libpython2.7.a: error adding symbols: Bad value
...

To fix this, please set the below environment variables before running 'pip install ...' For example, in bash:

    export CMAKE_INCLUDE_PATH=/root/path_of_python_installation/Python-3.6.2/include
    export CMAKE_LIBRARY_PATH=/root/path_of_python_installation/Python-3.6.2/lib

3) If facing issues with ydk installation in /usr/lib64 vs /usr/lib

See #524 for details and workaround

If facing issues with pip, another option is to use easy_install to install ydk-models-openconfig & other bundles

$ sudo easy_install -v --install-dir="/usr/lib64/python2.7/site-packages" ydk-models-ietf
$ sudo easy_install -v --install-dir="/usr/lib64/python2.7/site-packages" ydk-models-openconfig
$ sudo easy_install -v --install-dir="/usr/lib64/python2.7/site-packages" ydk-models-cisco-ios-xr

bmigette commented 3 years ago

devtools-4 are no longer available on centos 7.

Using default compiler gave me this error when importing ydk.services: undefined symbol: _ZSt24__throw_out_of_range_fmtPKcz

furthermore, --install-purelib option does not work with recent pip versions

Here's what worked for me: ` [root@controller ~]# yum install devtoolset-7-gcc* [root@controller ~]# ln -sf /opt/rh/devtoolset-7/root/usr/bin/gcc /usr/bin/cc [root@controller ~]# ln -sf /opt/rh/devtoolset-7/root/usr/bin/g++ /usr/bin/c++

pip3 install --target="/usr/local/lib64/python3.6/site-packages" --no-deps ydk-models-ietf ydk-models-openconfig ydk-models-cisco-ios-xr ydk-models-cisco-ios-xe --upgrade pip3 install ydk `

CiscoDevNet / ydk-gen