Python Install Issues

[dvedova]

Hello,

I recently tried to install idyntree following the installation guide in the readme in the root directory. After installing, I get an import error: ImportError: No module named iDynTree

This leads me to believe that I did not install idyntree correctly. Does anyone have any advice about this?

Attached is the CMakeLists.txt file I used to build iDynTree (I set IDYNTREE_USES_PYTHON to ON, but I'm not sure if this is the only thing that must be done).

Any advice or clarity on the python installation process would be much appreciated. Thanks!

CMakeLists.txt

[traversaro]

Hi @dvedova,
on which operating system you are? Where did you installed iDynTree (i.e. where did you set the CMAKE_INSTALL_PREFIX?) In general after you install a Python library not in an environment, you need to set the PYTHONPATH environment variable. I suspect we have a bug in our README at the moment, as we write:

export PYTHONPATH=$PYTHONPATH:<prefix>/lib/python2.7/dist-packages/

but in reality instead of python2.7 it should contained the python that you used for configuration. This has been fixed in the devel branch docs: https://github.com/robotology/idyntree/blob/devel/README.md#python , that will soon be released as iDynTree 2.0 (see https://github.com/robotology/idyntree/issues/772). Note that iDynTree 2.0 will contains some changes in the Python module name, see https://github.com/robotology/idyntree/blob/devel/CHANGELOG.md#changed .

If you are a Ubuntu user, we also have an experimental pip package that is still not documented, I guess it should work as:

sudo apt install libxml2-dev libassimp-dev libeigen3-dev coinor-libipopt-dev cmake ninja swig
# This should work also inside a virtualenv
pip3 install idyntree

but @diegoferigo can correct me if I am wrong.

[traversaro]

I also forgot to mention that depending on what you need we have a high-level more pythonic interface of iDynTree functionalities in https://robotology.github.io/gym-ignition/master/apidoc/gym-ignition/gym_ignition.rbd.idyntree.html#module-gym_ignition.rbd.idyntree.kindyncomputations, as part of the gym-ignition project (https://github.com/robotology/gym-ignition). It is still quite in development so there could be quirks, but you are welcome to try it.

[diegoferigo]

Very accurate recap @traversaro. In a plain system, the only missing piece is the swig dependency of the pip package.

[dvedova]

@traversaro Thanks for the quick reply.

I am running ubuntu 18.04.

I don't think I set the CMAKE_INSTALL_PREFIX to anything actually. The Idyntree directory is currently located and built in my home directory.

Should I set the CMAKE_INSTALL_PREFIX to the directory that contains my python?

I tried using pip to install, but I was unable to get that to work. Below is part of the error I get when I try to use pip3 install idyntree

RuntimeError: Required command 'ninja' not found

  ----------------------------------------
  Failed building wheel for idyntree

[traversaro]

I don't think I set the CMAKE_INSTALL_PREFIX to anything actually. The Idyntree directory is currently located and built in my home directory.

Not that to use the bindings, you don't just need to build the library, you also need to install, by running make install . While doing so, I personally suggest not to use the default value of CMAKE_INSTALL_PREFIX, that is /usr/local, as using it would mean that the iDynTree installation goes in a place where it could conflict with other software. Instead, you should set CMAKE_INSTALL_PREFIX to a directory inside your home, and you then you should modify the PYTHONPATH env variable to point to the place where the python bindings have been installed.

Should I set the CMAKE_INSTALL_PREFIX to the directory that contains my python?

No, that is not a great idea in general, unless you know exactly what you are doing.

I tried using pip to install, but I was unable to get that to work. Below is part of the error I get when I try to use pip3 install idyntree

Thanks, indeed there were an error in the docs reported in the comment. You also need to install the ninja-build apt package to get it to work:

sudo apt install ninja-build

[dvedova]

I installed ninja-build but this is the new error:

dvedova@danny-ThinkPad-P52s:~$ pip3 install idyntree
Collecting idyntree
  Using cached https://files.pythonhosted.org/packages/27/f0/927818a74f576d1a29225a4852f65f8ddc16e438f9d2a4026700f35ed8ef/idyntree-1.2.1.dev104.tar.gz
Collecting numpy (from idyntree)
  Using cached https://files.pythonhosted.org/packages/a6/fc/36e52d0ae2aa502b211f1bcd2fdeec72d343d58224eabcdddc1bcb052db1/numpy-1.19.4-cp36-cp36m-manylinux1_x86_64.whl
Building wheels for collected packages: idyntree
  Running setup.py bdist_wheel for idyntree ... error
  Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-93abb3f7/idyntree/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/tmp04t00_l6pip-wheel- --python-tag cp36:
  /usr/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'use_scm_version'
    warnings.warn(msg)
  running bdist_wheel
  running build
  running build_ext

  ==> Configuring:
  $ cmake -S /tmp/pip-build-93abb3f7/idyntree -B /tmp/pip-build-93abb3f7/idyntree/build/temp.linux-x86_64-3.6 -GNinja -DCMAKE_INSTALL_PREFIX:PATH=/tmp/pip-build-93abb3f7/idyntree/build/lib.linux-x86_64-3.6/idyntree -DBUILD_SHARED_LIBS:BOOL=OFF -DIDYNTREE_USES_PYTHON:BOOL=ON -DIDYNTREE_COMPILE_TESTS:BOOL=OFF -DIDYNTREE_PACKAGE_FOR_PYPI:BOOL=ON -DIDYNTREE_USES_IPOPT:BOOL=ON -DIDYNTREE_USES_ASSIMP:BOOL=ON -DIDYNTREE_USES_IRRLICHT:BOOL=OFF -DIDYNTREE_USES_QT5:BOOL=OFF -DIDYNTREE_USES_OSQPEIGEN:BOOL=OFF -DIDYNTREE_USES_ALGLIB:BOOL=OFF -DIDYNTREE_USES_WORHP:BOOL=OFF -DIDYNTREE_USES_YARP:BOOL=OFF -DIDYNTREE_USES_ICUB_MAIN:BOOL=OFF -DCMAKE_BUILD_TYPE=Release

  ==> Building:
  $ cmake --build /tmp/pip-build-93abb3f7/idyntree/build/temp.linux-x86_64-3.6 --config Release

  ==> Installing:
  $ cmake --build /tmp/pip-build-93abb3f7/idyntree/build/temp.linux-x86_64-3.6 --target install

  CMake Error: The source directory "/tmp/pip-build-93abb3f7/idyntree/build/temp.linux-x86_64-3.6" does not appear to contain CMakeLists.txt.
  Specify --help for usage, or press the help button on the CMake GUI.
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-build-93abb3f7/idyntree/setup.py", line 200, in <module>
      zip_safe=False,
    File "/home/dvedova/.local/lib/python3.6/site-packages/setuptools/__init__.py", line 144, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/home/dvedova/.local/lib/python3.6/site-packages/wheel/bdist_wheel.py", line 223, in run
      self.run_command('build')
    File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.6/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-93abb3f7/idyntree/setup.py", line 57, in run
      self.build_extension(ext)
    File "/tmp/pip-build-93abb3f7/idyntree/setup.py", line 144, in build_extension
      subprocess.check_call(configure_command)
    File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['cmake', '-S', '/tmp/pip-build-93abb3f7/idyntree', '-B', '/tmp/pip-build-93abb3f7/idyntree/build/temp.linux-x86_64-3.6', '-GNinja', '-DCMAKE_INSTALL_PREFIX:PATH=/tmp/pip-build-93abb3f7/idyntree/build/lib.linux-x86_64-3.6/idyntree', '-DBUILD_SHARED_LIBS:BOOL=OFF', '-DIDYNTREE_USES_PYTHON:BOOL=ON', '-DIDYNTREE_COMPILE_TESTS:BOOL=OFF', '-DIDYNTREE_PACKAGE_FOR_PYPI:BOOL=ON', '-DIDYNTREE_USES_IPOPT:BOOL=ON', '-DIDYNTREE_USES_ASSIMP:BOOL=ON', '-DIDYNTREE_USES_IRRLICHT:BOOL=OFF', '-DIDYNTREE_USES_QT5:BOOL=OFF', '-DIDYNTREE_USES_OSQPEIGEN:BOOL=OFF', '-DIDYNTREE_USES_ALGLIB:BOOL=OFF', '-DIDYNTREE_USES_WORHP:BOOL=OFF', '-DIDYNTREE_USES_YARP:BOOL=OFF', '-DIDYNTREE_USES_ICUB_MAIN:BOOL=OFF', '-DCMAKE_BUILD_TYPE=Release']' returned non-zero exit status 1.

  ----------------------------------------
  Failed building wheel for idyntree
  Running setup.py clean for idyntree
Failed to build idyntree
Installing collected packages: numpy, idyntree
  Running setup.py install for idyntree ... error
    Complete output from command /usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-93abb3f7/idyntree/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-fznny9bc-record/install-record.txt --single-version-externally-managed --compile --user --prefix=:
    /usr/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'use_scm_version'
      warnings.warn(msg)
    running install
    running build
    running build_ext

    ==> Configuring:
    $ cmake -S /tmp/pip-build-93abb3f7/idyntree -B /tmp/pip-build-93abb3f7/idyntree/build/temp.linux-x86_64-3.6 -GNinja -DCMAKE_INSTALL_PREFIX:PATH=/tmp/pip-build-93abb3f7/idyntree/build/lib.linux-x86_64-3.6/idyntree -DBUILD_SHARED_LIBS:BOOL=OFF -DIDYNTREE_USES_PYTHON:BOOL=ON -DIDYNTREE_COMPILE_TESTS:BOOL=OFF -DIDYNTREE_PACKAGE_FOR_PYPI:BOOL=ON -DIDYNTREE_USES_IPOPT:BOOL=ON -DIDYNTREE_USES_ASSIMP:BOOL=ON -DIDYNTREE_USES_IRRLICHT:BOOL=OFF -DIDYNTREE_USES_QT5:BOOL=OFF -DIDYNTREE_USES_OSQPEIGEN:BOOL=OFF -DIDYNTREE_USES_ALGLIB:BOOL=OFF -DIDYNTREE_USES_WORHP:BOOL=OFF -DIDYNTREE_USES_YARP:BOOL=OFF -DIDYNTREE_USES_ICUB_MAIN:BOOL=OFF -DCMAKE_BUILD_TYPE=Release

    ==> Building:
    $ cmake --build /tmp/pip-build-93abb3f7/idyntree/build/temp.linux-x86_64-3.6 --config Release

    ==> Installing:
    $ cmake --build /tmp/pip-build-93abb3f7/idyntree/build/temp.linux-x86_64-3.6 --target install

    CMake Error: The source directory "/tmp/pip-build-93abb3f7/idyntree/build/temp.linux-x86_64-3.6" does not appear to contain CMakeLists.txt.
    Specify --help for usage, or press the help button on the CMake GUI.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-93abb3f7/idyntree/setup.py", line 200, in <module>
        zip_safe=False,
      File "/home/dvedova/.local/lib/python3.6/site-packages/setuptools/__init__.py", line 144, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.6/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib/python3.6/distutils/dist.py", line 955, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/home/dvedova/.local/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/lib/python3.6/distutils/command/install.py", line 589, in run
        self.run_command('build')
      File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/usr/lib/python3.6/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/usr/lib/python3.6/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.6/distutils/dist.py", line 974, in run_command
        cmd_obj.run()
      File "/tmp/pip-build-93abb3f7/idyntree/setup.py", line 57, in run
        self.build_extension(ext)
      File "/tmp/pip-build-93abb3f7/idyntree/setup.py", line 144, in build_extension
        subprocess.check_call(configure_command)
      File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['cmake', '-S', '/tmp/pip-build-93abb3f7/idyntree', '-B', '/tmp/pip-build-93abb3f7/idyntree/build/temp.linux-x86_64-3.6', '-GNinja', '-DCMAKE_INSTALL_PREFIX:PATH=/tmp/pip-build-93abb3f7/idyntree/build/lib.linux-x86_64-3.6/idyntree', '-DBUILD_SHARED_LIBS:BOOL=OFF', '-DIDYNTREE_USES_PYTHON:BOOL=ON', '-DIDYNTREE_COMPILE_TESTS:BOOL=OFF', '-DIDYNTREE_PACKAGE_FOR_PYPI:BOOL=ON', '-DIDYNTREE_USES_IPOPT:BOOL=ON', '-DIDYNTREE_USES_ASSIMP:BOOL=ON', '-DIDYNTREE_USES_IRRLICHT:BOOL=OFF', '-DIDYNTREE_USES_QT5:BOOL=OFF', '-DIDYNTREE_USES_OSQPEIGEN:BOOL=OFF', '-DIDYNTREE_USES_ALGLIB:BOOL=OFF', '-DIDYNTREE_USES_WORHP:BOOL=OFF', '-DIDYNTREE_USES_YARP:BOOL=OFF', '-DIDYNTREE_USES_ICUB_MAIN:BOOL=OFF', '-DCMAKE_BUILD_TYPE=Release']' returned non-zero exit status 1.

    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-93abb3f7/idyntree/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-fznny9bc-record/install-record.txt --single-version-externally-managed --compile --user --prefix=" failed with error code 1 in /tmp/pip-build-93abb3f7/idyntree/

[traversaro]

Thanks @dvedova for your tests. I would say that something is missing in the pypi side, but @diegoferigo fors ure can provide more info.

[diegoferigo]

I think there is some confusion here, I'll try to get back on track. I'll assume it's ok for you installing the development version of idyntree (that corresponds to the devel branch).

You have two ways:

1. Use CMake: follow the official instructions being careful of enabling Python support, and then export PYTHONPATH to the folder inside the install prefix you selected as explained by @traversaro.
2. Use experimental pip support: this is what you did in the last comment. Note that it will download automatically in a temporary folder the sources of the git repository from PyPI, so following this way you don't need to clone anything.

Option 2 is preferred if you just need the Python bindings. Instead, option 1 is more flexible because it allows you to enable / disable components not included in the pip installation.

Let's proceed with the pip way, let me know then if you prefer the other way. I suspect you have a pip version too old that does not support PEP517 and PEP518. In fact, otherwise, ninja would be installed automatically by pip when building the idyntree wheel. The last error is another dependency that is installed automatically by recent versions of pip: setuptools_scm. Try to install this package manually and then repeat the pip install idyntree.

You can find the list of all the build dependencies in the pyproject.toml file.

[dvedova]

I went ahead and updated pip, and also did pip3 install setuptools_scm.

This is the new error when i command pip3 install idyntree:

dvedova@danny-ThinkPad-P52s:~/test_virtual$ pip3 install idyntree
WARNING: pip is being invoked by an old script wrapper. This will fail in a future version of pip.
Please see https://github.com/pypa/pip/issues/5599 for advice on fixing the underlying issue.
To avoid this problem you can invoke Python with '-m pip' instead of running pip directly.
Defaulting to user installation because normal site-packages is not writeable
Collecting idyntree
  Using cached idyntree-1.2.1.dev104.tar.gz (1.3 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting numpy
  Using cached numpy-1.19.4-cp38-cp38-manylinux2010_x86_64.whl (14.5 MB)
Building wheels for collected packages: idyntree
  Building wheel for idyntree (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 /home/dvedova/.local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /tmp/tmpr6jv9cby
       cwd: /tmp/pip-install-l630heew/idyntree
  Complete output (93 lines):
  running bdist_wheel
  running build
  running build_ext
  -- The C compiler identification is GNU 5.5.0
  -- The CXX compiler identification is GNU 5.5.0
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - done
  -- Check for working C compiler: /usr/bin/cc - skipped
  -- Detecting C compile features
  -- Detecting C compile features - done
  -- Detecting CXX compiler ABI info
  -- Detecting CXX compiler ABI info - done
  -- Check for working CXX compiler: /usr/bin/c++ - skipped
  -- Detecting CXX compile features
  -- Detecting CXX compile features - done
  -- Found LibXml2: /usr/lib/x86_64-linux-gnu/libxml2.so (found version "2.9.4")
  -- Found IPOPT: /usr/lib/libipopt.so;/usr/lib/x86_64-linux-gnu/libdmumps_seq.so;/usr/lib/x86_64-linux-gnu/libblas.so;/usr/lib/x86_64-linux-gnu/liblapack.so;/usr/lib/x86_64-linux-gnu/libblas.so;/usr/lib/x86_64-linux-gnu/libdmumps_seq.so;/usr/lib/x86_64-linux-gnu/libdl.so;/usr/lib/x86_64-linux-gnu/libm.so;/usr/lib/gcc/x86_64-linux-gnu/7/libquadmath.so;/usr/lib/x86_64-linux-gnu/libblas.so;/usr/lib/x86_64-linux-gnu/libm.so;/usr/lib/x86_64-linux-gnu/libdl.so
  -- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY
  -- Performing Test COMPILER_HAS_HIDDEN_VISIBILITY - Success
  -- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY
  -- Performing Test COMPILER_HAS_HIDDEN_INLINE_VISIBILITY - Success
  -- Performing Test COMPILER_HAS_DEPRECATED_ATTR
  -- Performing Test COMPILER_HAS_DEPRECATED_ATTR - Success
  -- Found SWIG: /usr/bin/swig3.0 (found version "3.0.12")
  -- Performing Test CXX_HAS_WNO_DEPRECATED_DECLARATIONS
  -- Performing Test CXX_HAS_WNO_DEPRECATED_DECLARATIONS - Success
  -- Performing Test CXX_HAS_WNO_DEPRECATED
  -- Performing Test CXX_HAS_WNO_DEPRECATED - Success
  CMake Error at /tmp/pip-build-env-cy4xpl6h/overlay/lib/python3.8/site-packages/cmake/data/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
    Could NOT find Python3 (missing: Python3_INCLUDE_DIRS Python3_LIBRARIES
    Development NumPy Development.Module Development.Embed) (found version
    "3.8.0")
  Call Stack (most recent call first):
    /tmp/pip-build-env-cy4xpl6h/overlay/lib/python3.8/site-packages/cmake/data/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:458 (_FPHSA_FAILURE_MESSAGE)
    /tmp/pip-build-env-cy4xpl6h/overlay/lib/python3.8/site-packages/cmake/data/share/cmake-3.18/Modules/FindPython/Support.cmake:2984 (find_package_handle_standard_args)
    /tmp/pip-build-env-cy4xpl6h/overlay/lib/python3.8/site-packages/cmake/data/share/cmake-3.18/Modules/FindPython3.cmake:389 (include)
    bindings/python/CMakeLists.txt:1 (find_package)

  -- Configuring incomplete, errors occurred!
  See also "/tmp/pip-install-l630heew/idyntree/build/temp.linux-x86_64-3.8/CMakeFiles/CMakeOutput.log".

  ==> Configuring:
  $ cmake -S /tmp/pip-install-l630heew/idyntree -B /tmp/pip-install-l630heew/idyntree/build/temp.linux-x86_64-3.8 -GNinja -DCMAKE_INSTALL_PREFIX:PATH=/tmp/pip-install-l630heew/idyntree/build/lib.linux-x86_64-3.8/idyntree -DBUILD_SHARED_LIBS:BOOL=OFF -DIDYNTREE_USES_PYTHON:BOOL=ON -DIDYNTREE_COMPILE_TESTS:BOOL=OFF -DIDYNTREE_PACKAGE_FOR_PYPI:BOOL=ON -DIDYNTREE_USES_IPOPT:BOOL=ON -DIDYNTREE_USES_ASSIMP:BOOL=ON -DIDYNTREE_USES_IRRLICHT:BOOL=OFF -DIDYNTREE_USES_QT5:BOOL=OFF -DIDYNTREE_USES_OSQPEIGEN:BOOL=OFF -DIDYNTREE_USES_ALGLIB:BOOL=OFF -DIDYNTREE_USES_WORHP:BOOL=OFF -DIDYNTREE_USES_YARP:BOOL=OFF -DIDYNTREE_USES_ICUB_MAIN:BOOL=OFF -DCMAKE_BUILD_TYPE=Release

  ==> Building:
  $ cmake --build /tmp/pip-install-l630heew/idyntree/build/temp.linux-x86_64-3.8 --config Release

  ==> Installing:
  $ cmake --build /tmp/pip-install-l630heew/idyntree/build/temp.linux-x86_64-3.8 --target install

  Traceback (most recent call last):
    File "/home/dvedova/.local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 280, in <module>
      main()
    File "/home/dvedova/.local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 263, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "/home/dvedova/.local/lib/python3.8/site-packages/pip/_vendor/pep517/_in_process.py", line 204, in build_wheel
      return _build_backend().build_wheel(wheel_directory, config_settings,
    File "/tmp/pip-build-env-cy4xpl6h/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 216, in build_wheel
      return self._build_with_temp_dir(['bdist_wheel'], '.whl',
    File "/tmp/pip-build-env-cy4xpl6h/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 202, in _build_with_temp_dir
      self.run_setup()
    File "/tmp/pip-build-env-cy4xpl6h/overlay/lib/python3.8/site-packages/setuptools/build_meta.py", line 145, in run_setup
      exec(compile(code, __file__, 'exec'), locals())
    File "setup.py", line 154, in <module>
      setup(
    File "/tmp/pip-build-env-cy4xpl6h/overlay/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
      return distutils.core.setup(**attrs)
    File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
      dist.run_commands()
    File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
      self.run_command(cmd)
    File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/tmp/pip-build-env-cy4xpl6h/overlay/lib/python3.8/site-packages/wheel/bdist_wheel.py", line 290, in run
      self.run_command('build')
    File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "/usr/lib/python3.8/distutils/command/build.py", line 135, in run
      self.run_command(cmd_name)
    File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
      self.distribution.run_command(command)
    File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
      cmd_obj.run()
    File "setup.py", line 57, in run
      self.build_extension(ext)
    File "setup.py", line 144, in build_extension
      subprocess.check_call(configure_command)
    File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
      raise CalledProcessError(retcode, cmd)
  subprocess.CalledProcessError: Command '['cmake', '-S', '/tmp/pip-install-l630heew/idyntree', '-B', '/tmp/pip-install-l630heew/idyntree/build/temp.linux-x86_64-3.8', '-GNinja', '-DCMAKE_INSTALL_PREFIX:PATH=/tmp/pip-install-l630heew/idyntree/build/lib.linux-x86_64-3.8/idyntree', '-DBUILD_SHARED_LIBS:BOOL=OFF', '-DIDYNTREE_USES_PYTHON:BOOL=ON', '-DIDYNTREE_COMPILE_TESTS:BOOL=OFF', '-DIDYNTREE_PACKAGE_FOR_PYPI:BOOL=ON', '-DIDYNTREE_USES_IPOPT:BOOL=ON', '-DIDYNTREE_USES_ASSIMP:BOOL=ON', '-DIDYNTREE_USES_IRRLICHT:BOOL=OFF', '-DIDYNTREE_USES_QT5:BOOL=OFF', '-DIDYNTREE_USES_OSQPEIGEN:BOOL=OFF', '-DIDYNTREE_USES_ALGLIB:BOOL=OFF', '-DIDYNTREE_USES_WORHP:BOOL=OFF', '-DIDYNTREE_USES_YARP:BOOL=OFF', '-DIDYNTREE_USES_ICUB_MAIN:BOOL=OFF', '-DCMAKE_BUILD_TYPE=Release']' returned non-zero exit status 1.
  ----------------------------------------
  ERROR: Failed building wheel for idyntree
Failed to build idyntree
ERROR: Could not build wheels for idyntree which use PEP 517 and cannot be installed directly

[traversaro]

This error seems that could be caused by the missing dev headers for Python and Numpy, I guess you also need to install python3-dev and python3-numpy (if you installed python via apt).

[diegoferigo]

The error of https://github.com/robotology/idyntree/issues/771#issuecomment-730455950 seems to be the only real bug of this issue, thanks for reporting.

I noticed you updated pip and python, wise choice. As you can see from the output of pip, numpy is actually installed in the temporary isolated environment created when building the wheel of idyntree. However, the CMake project fails to import it since its folder is not in the standard system search path.

This error seems that could be caused by the missing dev headers for Python and Numpy, I guess you also need to install python3-dev and python3-numpy (if you installed python via apt).

This can temporarily work, but I fear that it breaks all the purpose of having an isolated build. We already use some non-python dependency from the system (like libxml2, assimp, ...) but they are not python packages, so they are a valid exception. However, for fixing the installation of @dvedova this should suffice.

From our side, @traversaro, I think a better way to fix this problem is updating the configuration of the setuptools extension we are using by passing an extra f"-DPython3_NumPy_INCLUDE_DIRS={np.get_include()}" option. This way, setuptools should automatically detect the temporarily numpy installation inside the isolated environment created by pip.

[traversaro]

This can temporarily work, but I fear that it breaks all the purpose of having an isolated build.

We can probably have separate discussion for this. I guess the reason why you don't want to have that is that the pip-installed numpy may not be ABI-compatible with the numpy installed by apt, but what is the problem of using python3-dev if you are already using the python3 installed by apt?

[traversaro]

From our side, @traversaro, I think a better way to fix this problem is updating the configuration of the setuptools extension we are using by passing an extra f"-DPython3_NumPy_INCLUDE_DIRS={np.get_include()}" option. This way, setuptools should automatically detect the temporarily numpy installation inside the isolated environment created by pip.

But why this was not detected by the CI? Probably setup-python installs some packages under the hood?

[diegoferigo]

The CMake problem reported in the pip output of https://github.com/robotology/idyntree/issues/771#issuecomment-730455950 is:

  CMake Error at /tmp/pip-build-env-cy4xpl6h/overlay/lib/python3.8/site-packages/cmake/data/share/cmake-3.18/Modules/FindPackageHandleStandardArgs.cmake:165 (message):
    Could NOT find Python3 (missing: Python3_INCLUDE_DIRS Python3_LIBRARIES
    Development NumPy Development.Module Development.Embed) (found version
    "3.8.0")

Initially I suspected that there was a problem with numpy, but as @traversaro correctly wrote, our CI pipeline doesn't install numpy and still succeeds. Maybe the reason why Python is not found by CMake is the missing python3-dev package. If this is the case, forget my previous https://github.com/robotology/idyntree/issues/771#issuecomment-730476159.

This can temporarily work, but I fear that it breaks all the purpose of having an isolated build.

I guess the reason why you don't want to have that is that the pip-installed numpy may not be ABI-compatible with the numpy installed by apt, but what is the problem of using python3-dev if you are already using the python3 installed by apt?

My comment was about numpy, not python3-dev, sorry not being explicit in that. For what concern numpy, instead, it doesn't seem that the resulting swig bindings link dynamically against its libraries, so we should be ABI robust.

ldd

``` ❯ ldd /rl/venv/lib/python3.8/site-packages/idyntree/_iDynTree.so linux-vdso.so.1 (0x00007ffcd8af5000) libpython3.8.so.1.0 => /lib/x86_64-linux-gnu/libpython3.8.so.1.0 (0x00007faa5a999000) libassimp.so.5 => /lib/x86_64-linux-gnu/libassimp.so.5 (0x00007faa59d4c000) libxml2.so.2 => /lib/x86_64-linux-gnu/libxml2.so.2 (0x00007faa59b92000) libipopt.so.1 => /lib/libipopt.so.1 (0x00007faa59925000) libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007faa597d6000) libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007faa595f3000) libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007faa595d8000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007faa593e6000) libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007faa593b8000) libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007faa5939c000) libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007faa59379000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007faa59371000) libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007faa5936c000) libminizip.so.1 => /lib/x86_64-linux-gnu/libminizip.so.1 (0x00007faa59161000) libicuuc.so.66 => /lib/x86_64-linux-gnu/libicuuc.so.66 (0x00007faa58f7b000) liblzma.so.5 => /lib/x86_64-linux-gnu/liblzma.so.5 (0x00007faa58f52000) liblapack.so.3 => /lib/x86_64-linux-gnu/liblapack.so.3 (0x00007faa588ae000) libblas.so.3 => /lib/x86_64-linux-gnu/libblas.so.3 (0x00007faa5883f000) libdmumps_seq-5.2.1.so => /lib/x86_64-linux-gnu/libdmumps_seq-5.2.1.so (0x00007faa58630000) /lib64/ld-linux-x86-64.so.2 (0x00007faa5b469000) libicudata.so.66 => /lib/x86_64-linux-gnu/libicudata.so.66 (0x00007faa56b6f000) libgfortran.so.5 => /lib/x86_64-linux-gnu/libgfortran.so.5 (0x00007faa568a7000) libmumps_common_seq-5.2.1.so => /lib/x86_64-linux-gnu/libmumps_common_seq-5.2.1.so (0x00007faa5684f000) libquadmath.so.0 => /lib/x86_64-linux-gnu/libquadmath.so.0 (0x00007faa56803000) libpord_seq-5.2.1.so => /lib/x86_64-linux-gnu/libpord_seq-5.2.1.so (0x00007faa567ea000) libesmumps-6.so => /lib/x86_64-linux-gnu/libesmumps-6.so (0x00007faa567e4000) libscotch-6.so => /lib/x86_64-linux-gnu/libscotch-6.so (0x00007faa5674f000) libscotcherr-6.so => /lib/x86_64-linux-gnu/libscotcherr-6.so (0x00007faa5674a000) libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007faa56735000) ```

[dvedova]

So I was able use pip3 to install idyntree by downloading anaconda3. I'm not sure what else I did that led to this but I am happy with the result. Thank you @diegoferigo @traversaro for the timely assistance!

Here is the message from pip3 install idyntree:

(base) dvedova@danny-ThinkPad-P52s:~$ pip3 install idyntree
Collecting idyntree
  Using cached idyntree-1.2.1.dev104.tar.gz (1.3 MB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: numpy in ./anaconda3/lib/python3.8/site-packages (from idyntree) (1.18.5)
Building wheels for collected packages: idyntree
  Building wheel for idyntree (PEP 517) ... done
  Created wheel for idyntree: filename=idyntree-1.2.1.dev104-cp38-cp38-linux_x86_64.whl size=4148653 sha256=e4db311d17834afe477153afd3d3bf6d5b498033c5d8e71de9ce5174fda36a71
  Stored in directory: /home/dvedova/.cache/pip/wheels/5a/77/9d/86d8a2b3436695114d1467d4a44b8885e60ca30e5d97775b10
Successfully built idyntree
Installing collected packages: idyntree
Successfully installed idyntree-1.2.1.dev104

[traversaro]

So I was able use pip3 to install idyntree by downloading anaconda3. I'm not sure what else I did that led to this but I am happy with the result.

If you are using conda, to avoid conflicts make sure to install the iDynTree dependencies via conda, i.e.:

conda install -c conda-forge assimp libxml2 eigen ipopt swig

In general, pay attention with mixing pip, system dependencies and conda, because that leads to messed up situation you may not know how to debug. Furthermore, try to use environments as much as possible, i.e. if you use conda, don't install anything in the base environment.

[diegoferigo]

@dvedova I'm glad it went through!

Now that you managed to install iDynTree in your system, if you want a more simple and idiomatic way to use one of its main classes, KinDynComputations, you can have a look to the high-level wrapper we maintain in gym-ignition. It takes as inputs and returns as output numpy types, avoiding to deal with the SWIG types that are more difficult to use and require some experience.

Despite gym-ignition seems a tool only for reinforcement learning research, one of its components, Gazebo ScenarI/O, is comparable to other popular tools like mujoco-py and pybullet.

[dvedova]

Okay this is great! I am going to close this issue now. Thanks again.