Migrate Windows binary installer to be based on conda binary packages instead of vcpkg dependencies?

[traversaro]

Currently, for each release of the robotology-superbuild we are also shipping a Windows-installer built with QtIFW that installs binaries for almost all the packages of the robotology-superbuild. The binaries are build (in Debug and Release modes) by GitHub Actions (see https://github.com/robotology/robotology-superbuild/blob/v2021.08/.github/workflows/ci.yml#L462), except for dependencies that instead are built from vcpkg in another repo (in particular https://github.com/robotology/robotology-superbuild-dependencies-vcpkg). During the installation process, there is also an option (that by default is enabled) to set all the necessary environment variables for all the process created by the users (this is done by running the addPathsToUserEnvVariables.ps1 script during the installation process).

In the long run, I would like to avoid mantaining this vcpkg-based installation process, as it is quite tricky to debug if something goes wrong (a full build of the dependencies is ~8 hours, and then a full build of the installer given the binaries dependecies ~3 hours). As an alternative I wanted to start creating this installers using conda packages and the conda/constructor tool, see https://github.com/conda/constructor . This has both pros and cons, outlined in the following:

Pros of migrating the installer to be build using conda constructor:

• the main advantage (for me as the mantainer) that the mantainance burden is heavily reduced, as the conda constructor reuses the binaries already uploaded to the conda channels
• another great advantage (for users) is that the user after installing the installer, can eventually add on top of this installation other software, for example ROS packages provided by the RoboStack project, or PyTorch packages provided by conda-forge
• a difference, that however I think it is a pro for the user, is that all the environment variables would not be set at the global level, but rather only set if the terminal/environment of the installer is activated, reducing the risk of DLL hell

Cons of migrating the installer to be build using conda constructor:

• The main problem is that we don't ship Debug binaries in conda, so downstream user can't compile their own software that uses installer-provided libraries in Debug in MSVC (but they can still use RelWithDebInfo)
• ...

However, it is possible that there are more cons I am not aware of, and this is the reason why I opened this issue.

[traversaro]

Back in time I tought this installer were not used, but then recently I discovered indirectly that a few users where using them (in particular @sgiraz (that however I think already migrated to conda) , @Iaxama (due to https://github.com/robotology/robotology-superbuild/issues/608), @andrearosasco (due to https://github.com/robotology/yarp/issues/2710, even if I guess he already migrate to conda).

[traversaro]

I investigated a bit the use of constructor, and the use is relatively simple. You just need to put a file called construct.yaml in a directory, something like (local is for just created packages):

name: robotology-distro
version: 2022.02.0

channels:
  - local
  - conda-forge
  - robotology

specs:
  - conda
  - mamba
  - micromamba
  - robotology-distro-all 2022.02.0

Then go in that directory and execute:

constructor .

This creates a 1.1 GB robotology-distro-2022.02.0-Linux-x86_64.sh file, that we can then upload to the specific release. It should be relatively easy to add to the release CI.

[traversaro]

Some tests in https://github.com/robotology/robotology-superbuild/tree/addconstructor .

[traversaro]

Example:

WARNING: You need to make the package robotology-distro-all-1234.0-56.tar.bz2 available at robotology/linux-64/robotology-distro-all-1234.0-56.tar.bz2
Reading: /home/traversaro/mambaforge/envs/bldpkg/lib/python3.10/site-packages/constructor/header.sh
Successfully created '/home/traversaro/robotology-superbuild/buildpkg/conda/generated_constructor_recipe/robotology-distro-1234.0-Linux-x86_64.sh'.

[traversaro]

To support uploading, we can take inspiration from https://github.com/conda-forge/miniforge/blob/9d4744574dc7a246a3711cc1eaeb1202472f0c33/.github/workflows/ci.yml#L165 .