Upgrade to python 3.9 - Githubissues

mstoelzle commented 2 years ago

We upgrade to python 3.9, as there are still many packages missing for 3.10:

https://conda-forge.org/status/

Tobias-Fischer commented 2 years ago

The build numbers need to be bumped in all vinca_*.yaml files

Tobias-Fischer commented 2 years ago

There are several patches that need fixing up for Linux, are you happy to do that @mstoelzle?

On the other platforms we're running into #67 @wolfv

mstoelzle commented 2 years ago

There are several patches that need fixing up for Linux, are you happy to do that @mstoelzle?

On the other platforms we're running into #67 @wolfv

@Tobias-Fischer I can try next week, but I cannot promise anything.

I am still a bit confused by these patch files. Is there a documentation somewhere how you generate them? Is the purpose of them that you roll-back certain code snippets to older versions?

Tobias-Fischer commented 2 years ago

Hi @mstoelzle - the patches are needed as (unfortuantely) many packages do not compile out-of-the-box in RoboStack. The patches sometimes need updating when the packages get an update - in this case they might not apply anymore. Sometimes our pull requests get merged in, then we can simply remove the patch. Or sometimes we need to see what has changed and manually apply a similar change in the update version of the package.

There are at least two ways of creating them:

git clone the repository, change the file, and then git diff - this shows the patch. You can save it into a file using git diff > outfile.patch
Doing the change online on GitHub. If it's just a single file you are changing, you can then make the first steps toward a pull request. When the commit hash is shown in the URL, simply append .diff and it will show the diff. Or you can actually open a pull request and add .diff behind the pull request number - for this PR it would be https://github.com/RoboStack/ros-galactic/pull/75.diff

Hope that makes sense.

Tobias-Fischer commented 2 years ago

@wolfv - could you please have a look? All looks good to me.

Tobias-Fischer commented 2 years ago

Note: we should wait till the windows vinca issue is fixed

Tobias-Fischer commented 2 years ago

Needs https://github.com/RoboStack/vinca/pull/29

Tobias-Fischer commented 2 years ago

Fix #74 #79

Tobias-Fischer commented 2 years ago

Hi @mstoelzle - could you please give me write access to your fork?

Tobias-Fischer commented 2 years ago

Hi @wolfv - any chance you could have a look at this?

63
Preparing environment for ros2-distro-mutex
[64](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:64)

[65](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:65)
╭───────────────────── Traceback (most recent call last) ──────────────────────╮
[66](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:66)
│                                                                              │
[67](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:67)
│ /usr/local/miniconda/lib/python3.9/site-packages/boa/core/run_build.py:302   │
[68](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:68)
│ in build_recipe                                                              │
[69](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:69)
│                                                                              │
[70](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:70)
│   299 │   │   │   o.finalize_solve(sorted_outputs)                           │
[71](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:71)
│   300 │   │   │                                                              │
[72](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:72)
│   301 │   │   │   meta = MetaData(recipe_path, o)                            │
[73](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:73)
│ ❱ 302 │   │   │   o.set_final_build_id(meta)                                 │
[74](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:74)
│   303 │   │   │                                                              │
[75](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:75)
│   304 │   │   │   if o.skip() or full_render:                                │
[76](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:76)
│   305 │   │   │   │   continue                                               │
[77](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:77)
│ /usr/local/miniconda/lib/python3.9/site-packages/boa/core/recipe_output.py:5 │
[78](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:78)
│ 76 in set_final_build_id                                                     │
[79](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:79)
│                                                                              │
[80](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:80)
│   573 │   │   │   │   │   # remove self-run-exports for static packages      │
[81](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:81)
│   574 │   │   │   │   │   run_exports_list[:] = []                           │
[82](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:82)
│   575 │   │   │   │   else:                                                  │
[83](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:83)
│ ❱ 576 │   │   │   │   │   x.eval_pin_subpackage([self])                      │
[84](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:84)
│   577 │   │   │   │                                                          │
[85](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:85)
│   578 │   │   │   │   x.eval_pin_subpackage([self])                          │
[86](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:86)
│   579 │   │   │   run_exports_list[:] = [x.final for x in run_exports_list]  │
[87](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:87)
│                                                                              │
[88](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:88)
│ /usr/local/miniconda/lib/python3.9/site-packages/boa/core/conda_build_spec.p │
[89](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:89)
│ y:157 in eval_pin_subpackage                                                 │
[90](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:90)
│                                                                              │
[91](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:91)
│   154 │   │   version = output.version                                       │
[92](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:92)
│   155 │   │   build_string = output.final_build_id                           │
[93](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:93)
│   156 │   │                                                                  │
[94](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:94)
│ ❱ 157 │   │   if self.pin.exact:                                             │
[95](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:95)
│   158 │   │   │   self.final = f"{pkg_name} {version} {build_string}"        │
[96](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:96)
│   159 │   │   else:                                                          │
[97](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:97)
│   160 │   │   │   version_parts = version.split(".")                         │
[98](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:98)
╰──────────────────────────────────────────────────────────────────────────────╯
[99](https://github.com/RoboStack/ros-galactic/runs/5653129924?check_suite_focus=true#step:16:99)
AttributeError: 'NoneType' object has no attribute 'exact'

mstoelzle commented 2 years ago

Hi @mstoelzle - could you please give me write access to your fork?

Yes, I just added you as a collaborator to the fork

Tobias-Fischer commented 2 years ago

Any ideas what could be going wrong on Win @wolfv @traversaro?

The system cannot find the file specified.
CMake Error: File C:/Users/runneradmin/AppData/Local/Temp/tmp7_771aunlocal_setup.bat.in does not exist.
CMake Error at cmake/environment_hooks/ament_generate_package_environment.cmake:73 (configure_file):
  configure_file Problem configuring file
Call Stack (most recent call first):
  cmake/environment_hooks/ament_cmake_environment_hooks_package_hook.cmake:26 (ament_generate_package_environment)
  cmake/core/ament_execute_extensions.cmake:48 (include)
  cmake/core/ament_package.cmake:66 (ament_execute_extensions)
  CMakeLists.txt:21 (ament_package)

Tobias-Fischer commented 2 years ago

Seems like a slash is missing, but I can't figure out where

traversaro commented 2 years ago

Any ideas what could be going wrong on Win @wolfv @traversaro?

The system cannot find the file specified.
CMake Error: File C:/Users/runneradmin/AppData/Local/Temp/tmp7_771aunlocal_setup.bat.in does not exist.
CMake Error at cmake/environment_hooks/ament_generate_package_environment.cmake:73 (configure_file):
  configure_file Problem configuring file
Call Stack (most recent call first):
  cmake/environment_hooks/ament_cmake_environment_hooks_package_hook.cmake:26 (ament_generate_package_environment)
  cmake/core/ament_execute_extensions.cmake:48 (include)
  cmake/core/ament_package.cmake:66 (ament_execute_extensions)
  CMakeLists.txt:21 (ament_package)

I guess the issue is something around https://github.com/ament/ament_cmake/blob/1.3.0/ament_cmake_core/cmake/package_templates/templates_2_cmake.py#L84, but it is difficult to debug without running it locally. Do you have any suggestion on the best way of reproduce this locally?

Tobias-Fischer commented 2 years ago

You should be able to follow the workflow in https://github.com/mstoelzle/ros-galactic/blob/python39/.github/workflows/testpr.yml

Specifically:

mamba create -n testenv python=3.9 pip rospkg networkx "catkin_pkg>=0.4.16" "ruamel.yaml>=0.16.6" "rosdistro>=0.8.0" "empy>=3.3.4" boa=0.9
mamba activate testenv
pip install git+https://github.com/RoboStack/vinca.git --no-deps
cd /path/to/ros-galactic
cp vinca_win.yaml vinca.yaml
vinca --platform win-64
boa build additional_recipes/ros2-distro-mutex/ -m ./.ci_support/conda_forge_pinnings.yaml -m ./conda_build_config.yaml
boa build . -m ./.ci_support/conda_forge_pinnings.yaml -m ./conda_build_config.yaml

Tobias-Fischer commented 2 years ago

All other platforms seem fine now - linux-aarch64 and osx-arm64 builds are only broken as the native builds of cyclonedds are needed.

Tobias-Fischer commented 2 years ago

Merging in - let's fix Windows separately

RoboStack / ros-galactic

Upgrade to python 3.9 #75