Full Rebuild January 2024 with updated conda-forge pinnings

[traversaro]

Just a test for now, but it should solve several failures that we recently saw due to conflicts with conda-forge abi migrations, while robostack package still use older version of the dependencies.

I used the pinning used as of 2023/10/14 in https://github.com/conda-forge/conda-forge-pinning-feedstock/commit/932c544224ba478780043bad3c8ac22abc59cca3, except for libopencv and bullet-cpp that I manually bumped as they both have abi migration that are almost complete.

Fix https://github.com/RoboStack/ros-noetic/issues/366 . Fix https://github.com/RoboStack/ros-noetic/issues/415 .

[traversaro]

linux-aarch64 failed early with error:

2023-10-14T10:45:54.4324552Z subprocess.CalledProcessError: Command '['/home/runner/micromamba/envs/testpr_env/bin/patch', '--no-backup-if-mismatch', '--batch', '-Np1', '-i', '/tmp/tmpucm75b5w/ros-noetic-foxglove-bridge.patch.native', '--binary', '--dry-run']' returned non-zero exit status 1.

[Tobias-Fischer]

Thanks for taking the initiative, @traversaro!

Regarding pinnings, some more candidates:

• libboost=182
• ffmpeg=6
• icu=73
• libjpeg_turbo=3
• proj=930
• vtk=926

I'll probably have time to help out in a few weeks time once the semester is over. I found that it is fastest to iterate by locally building first, then try building in CI.

How about a bump of Python as well? At least to 3.11, potentially even 3.12?

[traversaro]

I found that it is fastest to iterate by locally building first, then try building in CI.

Sure! I was just confused on how to replicate the CI locally, but after following the steps in the CI scripts locally, now I was able to replicate them.

How about a bump of Python as well? At least to 3.11, potentially even 3.12?

I am not sure about this, at the moment one of my main goals were fixing a few problems with Python 3.9 packages, once the build works fine probably we can think if either get a full rebuild with Python 3.9 packages, and after update, or how to proceed.

[traversaro]

Regarding pinnings, some more candidates:

* libboost=182

* ffmpeg=6

* icu=73

* libjpeg_turbo=3

* proj=930

* vtk=926

ffmpeg=6 is already done in the updated conda_forge pinnings, for the others I can add them in the local config!

[traversaro]

CI builds fail with:

2023-10-15T16:07:56.7150459Z with channels:
2023-10-15T16:07:56.7150655Z 
2023-10-15T16:07:56.7150788Z The reported errors are:
2023-10-15T16:07:56.7151581Z - Encountered problems while solving:
2023-10-15T16:07:56.7152827Z -   - package ros-noetic-eigenpy-2.8.1-hb0f4dca_20 requires eigenpy 2.8.1*, but none of the providers can be installed
2023-10-15T16:07:56.7153834Z - 

Somehow, the updated additional package for ros-noetic-eigenpy is not build. Manually building locally make the build go a lot forward, even if there are build failures for hpp-fcl and pinocchio. To solve them, I also converted them to additional packages that only depend on the conda-forge versions of the packages. Probably it would be a good time to work on a slightly better solution for https://github.com/RoboStack/robostack.github.io/issues/17 .

[traversaro]

macos builds are failing with:

Finalizing build for ros-noetic-catkin
/Users/runner/work/_temp/08e55ca9-7b5f-4243-9467-f32236fb354f.sh: line 2:  2523 Segmentation fault: 11  boa build . -m ./.ci_support/conda_forge_pinnings.yaml -m ./conda_build_config.yaml
/Users/runner/micromamba/envs/testpr_env/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
Error: Process completed with exit code 139.

Windows builds are failing with:

2023-10-16T08:43:19.4350412Z Preparing environment for ros-noetic-catkin
2023-10-16T08:43:19.4351169Z 
2023-10-16T08:43:19.4351378Z Finalizing build for ros-noetic-catkin
2023-10-16T08:43:19.5985555Z D:\a\_temp\ab4ceee5-fd41-4bd5-be37-adf670e090df.sh: line 2:  1482 Segmentation fault      boa build . -m ./.ci_support/conda_forge_pinnings.yaml -m ./conda_build_config.yaml
2023-10-16T08:43:19.6020421Z ##[error]Process completed with exit code 139.
2023-10-16T08:43:19.6258111Z Post job cleanup.

I tried to reproduce the Windows failure locally without success.

[Tobias-Fischer]

These segfaults might be from memory leaks or similar in boa. They often go away in the actual CI on main

[traversaro]

linux-aarch64 fails with:

2023-10-16T10:37:08.5922205Z [1/17] Generating __/__/include/qt_gui_cpp/moc_plugin_bridge.cpp
2023-10-16T10:37:08.5923760Z FAILED: src/qt_gui_cpp/__/__/include/qt_gui_cpp/moc_plugin_bridge.cpp $SRC_DIR/build/src/qt_gui_cpp/__/__/include/qt_gui_cpp/moc_plugin_bridge.cpp 
2023-10-16T10:37:08.5925775Z cd $SRC_DIR/build/src/qt_gui_cpp/__/__/include/qt_gui_cpp && $PREFIX/bin/moc @$SRC_DIR/build/src/qt_gui_cpp/__/__/include/qt_gui_cpp/moc_plugin_bridge.cpp_parameters
2023-10-16T10:37:08.5927148Z /bin/sh: 1: $PREFIX/bin/moc: Exec format error
2023-10-16T10:37:08.5927851Z [2/17] Generating __/__/include/qt_gui_cpp/moc_plugin.cpp
2023-10-16T10:37:08.5929263Z FAILED: src/qt_gui_cpp/__/__/include/qt_gui_cpp/moc_plugin.cpp $SRC_DIR/build/src/qt_gui_cpp/__/__/include/qt_gui_cpp/moc_plugin.cpp 
2023-10-16T10:37:08.5931241Z cd $SRC_DIR/build/src/qt_gui_cpp/__/__/include/qt_gui_cpp && $PREFIX/bin/moc @$SRC_DIR/build/src/qt_gui_cpp/__/__/include/qt_gui_cpp/moc_plugin.cpp_parameters
2023-10-16T10:37:08.5932576Z /bin/sh: 1: $PREFIX/bin/moc: Exec format error
2023-10-16T10:37:08.5933313Z [3/17] Generating __/__/include/qt_gui_cpp/moc_plugin_context.cpp
2023-10-16T10:37:10.5635208Z FAILED: src/qt_gui_cpp/__/__/include/qt_gui_cpp/moc_plugin_context.cpp $SRC_DIR/build/src/qt_gui_cpp/__/__/include/qt_gui_cpp/moc_plugin_context.cpp 
2023-10-16T10:37:10.5637552Z cd $SRC_DIR/build/src/qt_gui_cpp/__/__/include/qt_gui_cpp && $PREFIX/bin/moc @$SRC_DIR/build/src/qt_gui_cpp/__/__/include/qt_gui_cpp/moc_plugin_context.cpp_parameters
2023-10-16T10:37:10.5639136Z /bin/sh: 1: $PREFIX/bin/moc: Exec format error
2023-10-16T10:37:10.5639920Z ninja: build stopped: subcommand failed.

That as far as I remember was a issue we had also in the past, and we solved by locally building a package.

linux-64 fails with:

2023-10-16T10:28:31.3891842Z ╰──────────────────────────────────────────────────────────────────────────────╯
2023-10-16T10:28:31.3892605Z RuntimeError: Solver could not find solution.Mamba failed to solve:
2023-10-16T10:28:31.3893335Z  - assimp 5.3.1*
2023-10-16T10:28:31.3893698Z  - boost 1.82.0*
2023-10-16T10:28:31.3894045Z  - eigen
2023-10-16T10:28:31.3894358Z  - graphviz 8.*
2023-10-16T10:28:31.3894667Z  - lxml
2023-10-16T10:28:31.3894968Z  - numpy 1.22.*
2023-10-16T10:28:31.3895287Z  - pip
2023-10-16T10:28:31.3895571Z  - pkg-config
2023-10-16T10:28:31.3895923Z  - python 3.9.* *_cpython
2023-10-16T10:28:31.3896353Z  - ros-distro-mutex 0.5.* noetic
2023-10-16T10:28:31.3896780Z  - ros-noetic-eigenpy
2023-10-16T10:28:31.3897162Z  - ros-noetic-octomap
2023-10-16T10:28:31.3897520Z  - doxygen
2023-10-16T10:28:31.3897798Z  - git
2023-10-16T10:28:31.3898111Z  - libstdcxx-ng >=12
2023-10-16T10:28:31.3898475Z  - libgcc-ng >=12
2023-10-16T10:28:31.3898823Z  - libgcc-ng >=12
2023-10-16T10:28:31.3899184Z  - __glibc >=2.17,<3.0.a0
2023-10-16T10:28:31.3899427Z 
2023-10-16T10:28:31.3899539Z with channels:
2023-10-16T10:28:31.3899742Z 
2023-10-16T10:28:31.3899868Z The reported errors are:
2023-10-16T10:28:31.3900314Z - Encountered problems while solving:
2023-10-16T10:28:31.3901106Z -   - package ros-noetic-eigenpy-2.8.1-hb0f4dca_20 requires eigenpy 2.8.1*, but 
2023-10-16T10:28:31.3901811Z none of the providers can be installed
2023-10-16T10:28:31.3902252Z - 

I solved this locally by manually building the ros-noetic-eigenpy package from additional_recipes, not sure if there is something we could do to build them automatically (perhaps removing eigenpy from the packages_skip_by_deps section)?

[Tobias-Fischer]

On the main branch, we have a native aarch64 build via https://github.com/RoboStack/ros-noetic/blob/master/.cirun.yml so the error should go away.

For linux-64, I temporarily enabled building the ros-noetic-eigenpy package; unfortunately there is no neat solution that checks the build number.

[Tobias-Fischer]

I think run_exports does not work as intended with boa (building ros-noetic-eigenpy fails):

AttributeError: 'str' object has no attribute 'final_pin'

[traversaro]

I think run_exports does not work as intended with boa (building ros-noetic-eigenpy fails):

AttributeError: 'str' object has no attribute 'final_pin'

I was able to fix the problems by changing the structure of the recipe, see https://github.com/RoboStack/ros-noetic/pull/418/commits/2c74a1b3138dfb33b374654d954942280d4e8145 .

[Tobias-Fischer]

@traversaro @wolfv - any chance to give this a quick review, please?

[traversaro]

@traversaro @wolfv - any chance to give this a quick review, please?

Everything seems fine, I just have a comment inline but non blocking.