search

robotology / robotology-superbuild

CMake/YCM-based superbuild to simplify the build process of robotology projects.
49 stars 48 forks source link

Build Python bindings of manifpy on Windows and switch manif version to 0.0.5 #1701

Closed traversaro closed 3 weeks ago

traversaro commented 4 weeks ago

Remove workaround added in https://github.com/robotology/robotology-superbuild/pull/838#issuecomment-887367047 .

traversaro commented 4 weeks ago

Hopefully this will work out of the box, if not we can also try to update manif to 0.0.5, this was already tested in https://github.com/ami-iit/bipedal-locomotion-framework/pull/874 .

giotherobot commented 4 weeks ago

Building on windows results in: image Sorry for image...

traversaro commented 4 weeks ago

Ok, we see the same error in CI:

2024-09-04T10:31:21.0311241Z C:\PROGRA~2\MICROS~2\2019\ENTERP~1\VC\Tools\MSVC\1429~1.301\bin\Hostx64\x64\cl.exe  /nologo /TP -DEIGEN_DEFAULT_TO_ROW_MAJOR -Dmanifpy_EXPORTS -ID:\a\robotology-superbuild\robotology-superbuild\src\manif\include -ID:\a\robotology-superbuild\robotology-superbuild\src\manif\external\tl -external:ID:\a\robotology-superbuild\robotology-superbuild\.pixi\envs\default\Library\include -external:ID:\a\robotology-superbuild\robotology-superbuild\.pixi\envs\default\Include -external:ID:\a\robotology-superbuild\robotology-superbuild\.pixi\envs\default\Library\include\eigen3 -external:W0 /DWIN32 /D_WINDOWS /W3 /GR /EHsc /MD /O2 /Ob2 /DNDEBUG -std:c++17 /GL /bigobj /MP /showIncludes /Fopython\CMakeFiles\manifpy.dir\bindings_so2.cpp.obj /Fdpython\CMakeFiles\manifpy.dir\ /FS -c D:\a\robotology-superbuild\robotology-superbuild\src\manif\python\bindings_so2.cpp
2024-09-04T10:31:21.0319107Z D:\a\robotology-superbuild\robotology-superbuild\.pixi\envs\default\Library\include\pybind11/pybind11.h(1669): error C2661: 'pybind11::cpp_function::cpp_function': no overloaded function takes 8 arguments
2024-09-04T10:31:21.0323530Z D:\a\robotology-superbuild\robotology-superbuild\src\manif\python\bindings_lie_group_base.h(5): note: see reference to function template instantiation 'pybind11::class_<manif::SO2d,manif::LieGroupBase<_Derived>> &pybind11::class_<manif::SO2d,manif::LieGroupBase<_Derived>>::def<manif::SO2<double>(__cdecl manif::SO2<double>::* )(const manif::LieGroupBase<_Derived>
traversaro commented 4 weeks ago

Let's try to update to 0.0.5 at this point.

traversaro commented 4 weeks ago

Let's try to update to 0.0.5 at this point.

Done in https://github.com/robotology/robotology-superbuild/pull/1701/commits/950214fa4c6a81ff30006e880bd9db822450d86b . You can do it manually to test by switching the remote to https://github.com/artivis/manif and the tag to 0.0.5 .

traversaro commented 4 weeks ago

Also with 0.0.5 the build fails with:

[6/9] Building CXX object python\CMakeFiles\manifpy.dir\bindings_rn.cpp.obj
FAILED: python/CMakeFiles/manifpy.dir/bindings_rn.cpp.obj 
C:\PROGRA~2\MICROS~2\2019\ENTERP~1\VC\Tools\MSVC\1429~1.301\bin\Hostx64\x64\cl.exe  /nologo /TP -DEIGEN_DEFAULT_TO_ROW_MAJOR -Dmanifpy_EXPORTS -ID:\a\robotology-superbuild\robotology-superbuild\src\manif\include -ID:\a\robotology-superbuild\robotology-superbuild\src\manif\external\tl -external:ID:\a\robotology-superbuild\robotology-superbuild\.pixi\envs\default\Library\include -external:ID:\a\robotology-superbuild\robotology-superbuild\.pixi\envs\default\Include -external:ID:\a\robotology-superbuild\robotology-superbuild\.pixi\envs\default\Library\include\eigen3 -external:W0 /DWIN32 /D_WINDOWS /W3 /GR /EHsc /MD /O2 /Ob2 /DNDEBUG -std:c++17 /GL /bigobj /MP /showIncludes /Fopython\CMakeFiles\manifpy.dir\bindings_rn.cpp.obj /Fdpython\CMakeFiles\manifpy.dir\ /FS -c D:\a\robotology-superbuild\robotology-superbuild\src\manif\python\bindings_rn.cpp
D:\a\robotology-superbuild\robotology-superbuild\.pixi\envs\default\Library\include\pybind11/pybind11.h(1669): error C2661: 'pybind11::cpp_function::cpp_function': no overloaded function takes 8 arguments
D:\a\robotology-superbuild\robotology-superbuild\src\manif\python\bindings_lie_group_base.h(5): note: see reference to function template instantiation 'pybind11::class_<manif::R1d,manif::LieGroupBase<manif::R1d>> &pybind11::class_<manif::R1d,manif::LieGroupBase<manif::R1d>>::def<manif::Rn<double,1>(__cdecl manif::Rn<double,1>::* )(const manif::LieGroupBase<manif::R1d> &,tl::optional<Eigen::Ref<Eigen::Matrix<double,1,1,1,1,1>,0,Eigen::InnerStride<1>>>,tl::optional<Eigen::Ref<Eigen::Matrix<double,1,1,1,1,1>,0,Eigen::InnerStride<1>>>) const,pybind11::arg,pybind11::arg_v,pybind11::arg_v,char[358]>(const char *,Func &&,const pybind11::arg &,const pybind11::arg_v &,const pybind11::arg_v &,const char (&)[358])' being compiled
        with
        [
            Func=manif::Rn<double,1> (__cdecl manif::Rn<double,1>::* )(const manif::LieGroupBase<manif::R1d> &,tl::optional<Eigen::Ref<Eigen::Matrix<double,1,1,1,1,1>,0,Eigen::InnerStride<1>>>,tl::optional<Eigen::Ref<Eigen::Matrix<double,1,1,1,1,1>,0,Eigen::InnerStride<1>>>) const
        ]
D:\a\robotology-superbuild\robotology-superbuild\src\manif\python\bindings_lie_group_base.h(104): note: see reference to function template instantiation 'pybind11::class_<manif::R1d,manif::LieGroupBase<manif::R1d>> &pybind11::class_<manif::R1d,manif::LieGroupBase<manif::R1d>>::def<manif::Rn<double,1>(__cdecl manif::Rn<double,1>::* )(const manif::LieGroupBase<manif::R1d> &,tl::optional<Eigen::Ref<Eigen::Matrix<double,1,1,1,1,1>,0,Eigen::InnerStride<1>>>,tl::optional<Eigen::Ref<Eigen::Matrix<double,1,1,1,1,1>,0,Eigen::InnerStride<1>>>) const,pybind11::arg,pybind11::arg_v,pybind11::arg_v,char[358]>(const char *,Func &&,const pybind11::arg &,const pybind11::arg_v &,const pybind11::arg_v &,const char (&)[358])' being compiled
        with
        [
            Func=manif::Rn<double,1> (__cdecl manif::Rn<double,1>::* )(const manif::LieGroupBase<manif::R1d> &,tl::optional<Eigen::Ref<Eigen::Matrix<double,1,1,1,1,1>,0,Eigen::InnerStride<1>>>,tl::optional<Eigen::Ref<Eigen::Matrix<double,1,1,1,1,1>,0,Eigen::InnerStride<1>>>) const
        ]
D:\a\robotology-superbuild\robotology-superbuild\src\manif\python\bindings_rn.cpp(23): note: see reference to function template instantiation 'void wrap_lie_group_base<manif::R1d,manif::LieGroupBase<manif::R1d>>(pybind11::class_<manif::R1d,manif::LieGroupBase<manif::R1d>> &)' being compiled

ninja: build stopped: subcommand failed.
traversaro commented 4 weeks ago

The only reference to a similar error I can find is https://github.com/emlearn/emlearn/issues/40 .

traversaro commented 4 weeks ago

Also with 0.0.5 the build fails with:

[6/9] Building CXX object python\CMakeFiles\manifpy.dir\bindings_rn.cpp.obj
FAILED: python/CMakeFiles/manifpy.dir/bindings_rn.cpp.obj 
C:\PROGRA~2\MICROS~2\2019\ENTERP~1\VC\Tools\MSVC\1429~1.301\bin\Hostx64\x64\cl.exe  /nologo /TP -DEIGEN_DEFAULT_TO_ROW_MAJOR -Dmanifpy_EXPORTS -ID:\a\robotology-superbuild\robotology-superbuild\src\manif\include -ID:\a\robotology-superbuild\robotology-superbuild\src\manif\external\tl -external:ID:\a\robotology-superbuild\robotology-superbuild\.pixi\envs\default\Library\include -external:ID:\a\robotology-superbuild\robotology-superbuild\.pixi\envs\default\Include -external:ID:\a\robotology-superbuild\robotology-superbuild\.pixi\envs\default\Library\include\eigen3 -external:W0 /DWIN32 /D_WINDOWS /W3 /GR /EHsc /MD /O2 /Ob2 /DNDEBUG -std:c++17 /GL /bigobj /MP /showIncludes /Fopython\CMakeFiles\manifpy.dir\bindings_rn.cpp.obj /Fdpython\CMakeFiles\manifpy.dir\ /FS -c D:\a\robotology-superbuild\robotology-superbuild\src\manif\python\bindings_rn.cpp
D:\a\robotology-superbuild\robotology-superbuild\.pixi\envs\default\Library\include\pybind11/pybind11.h(1669): error C2661: 'pybind11::cpp_function::cpp_function': no overloaded function takes 8 arguments
D:\a\robotology-superbuild\robotology-superbuild\src\manif\python\bindings_lie_group_base.h(5): note: see reference to function template instantiation 'pybind11::class_<manif::R1d,manif::LieGroupBase<manif::R1d>> &pybind11::class_<manif::R1d,manif::LieGroupBase<manif::R1d>>::def<manif::Rn<double,1>(__cdecl manif::Rn<double,1>::* )(const manif::LieGroupBase<manif::R1d> &,tl::optional<Eigen::Ref<Eigen::Matrix<double,1,1,1,1,1>,0,Eigen::InnerStride<1>>>,tl::optional<Eigen::Ref<Eigen::Matrix<double,1,1,1,1,1>,0,Eigen::InnerStride<1>>>) const,pybind11::arg,pybind11::arg_v,pybind11::arg_v,char[358]>(const char *,Func &&,const pybind11::arg &,const pybind11::arg_v &,const pybind11::arg_v &,const char (&)[358])' being compiled
        with
        [
            Func=manif::Rn<double,1> (__cdecl manif::Rn<double,1>::* )(const manif::LieGroupBase<manif::R1d> &,tl::optional<Eigen::Ref<Eigen::Matrix<double,1,1,1,1,1>,0,Eigen::InnerStride<1>>>,tl::optional<Eigen::Ref<Eigen::Matrix<double,1,1,1,1,1>,0,Eigen::InnerStride<1>>>) const
        ]
D:\a\robotology-superbuild\robotology-superbuild\src\manif\python\bindings_lie_group_base.h(104): note: see reference to function template instantiation 'pybind11::class_<manif::R1d,manif::LieGroupBase<manif::R1d>> &pybind11::class_<manif::R1d,manif::LieGroupBase<manif::R1d>>::def<manif::Rn<double,1>(__cdecl manif::Rn<double,1>::* )(const manif::LieGroupBase<manif::R1d> &,tl::optional<Eigen::Ref<Eigen::Matrix<double,1,1,1,1,1>,0,Eigen::InnerStride<1>>>,tl::optional<Eigen::Ref<Eigen::Matrix<double,1,1,1,1,1>,0,Eigen::InnerStride<1>>>) const,pybind11::arg,pybind11::arg_v,pybind11::arg_v,char[358]>(const char *,Func &&,const pybind11::arg &,const pybind11::arg_v &,const pybind11::arg_v &,const char (&)[358])' being compiled
        with
        [
            Func=manif::Rn<double,1> (__cdecl manif::Rn<double,1>::* )(const manif::LieGroupBase<manif::R1d> &,tl::optional<Eigen::Ref<Eigen::Matrix<double,1,1,1,1,1>,0,Eigen::InnerStride<1>>>,tl::optional<Eigen::Ref<Eigen::Matrix<double,1,1,1,1,1>,0,Eigen::InnerStride<1>>>) const
        ]
D:\a\robotology-superbuild\robotology-superbuild\src\manif\python\bindings_rn.cpp(23): note: see reference to function template instantiation 'void wrap_lie_group_base<manif::R1d,manif::LieGroupBase<manif::R1d>>(pybind11::class_<manif::R1d,manif::LieGroupBase<manif::R1d>> &)' being compiled

ninja: build stopped: subcommand failed.

I reported the problem upstream in https://github.com/artivis/manif/issues/311, and tried to add a workaround in https://github.com/robotology/robotology-superbuild/pull/1701/commits/f064809311dff1c8ec23dd3fe4177843fb6f5bd0 that relies on https://github.com/robotology/ycm-cmake-modules/pull/461 .

traversaro commented 3 weeks ago

Great, the solution worked great when using the Visual Studio generator and setting CMAKE_GENERATOR_TOOLSET to ClangCL, validating also https://github.com/robotology/ycm-cmake-modules/pull/461 . The solution when ninja is usesd does not work as just passing the clang-cl.exe as C/C++ compiler CMake ends up picking the really old clang 5 that gets installed on Windows if you install the compilers meta package, via the old classic flang package. This is yet another thing that will be solved by https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/1359, but for now we can hardcode the path to the clang-cl shipped with visual studio (the same used by ClangCL's CMAKE_GENERATOR_TOOLSET) and hopefully everything should work.

traversaro commented 3 weeks ago

Image for docs:

clangclinstall

clangclinstall2019

traversaro commented 3 weeks ago

Great, the solution worked great when using the Visual Studio generator and setting CMAKE_GENERATOR_TOOLSET to ClangCL, validating also robotology/ycm-cmake-modules#461 . The solution when ninja is usesd does not work as just passing the clang-cl.exe as C/C++ compiler CMake ends up picking the really old clang 5 that gets installed on Windows if you install the compilers meta package, via the old classic flang package. This is yet another thing that will be solved by conda-forge/conda-forge-pinning-feedstock#1359, but for now we can hardcode the path to the clang-cl shipped with visual studio (the same used by ClangCL's CMAKE_GENERATOR_TOOLSET) and hopefully everything should work.

Ok, I did this and it seems to work.

traversaro commented 3 weeks ago

Ok, I also added some docs in https://github.com/robotology/robotology-superbuild/pull/1701/commits/7625d46439b9d8b03d3365ca8fea2f2dead4d48b, the PR is ready for review.

traversaro commented 3 weeks ago

fyi @GiulioRomualdi this changes the version of manif to 0.0.5, that has been already tested in blf CI in https://github.com/ami-iit/bipedal-locomotion-framework/pull/874 .