Is a GitHub action really necessary to build a C++ based project?

lukka commented 3 years ago

Introduction

In an ideal world, a mature ecosystem around the 30 years old C++ language would have standard and multi platform tools handling the dependency management, the building, the testing and so forth. Such as:

cpp install && cpp build && cpp test

Ideally those exact commands would be run locally on the developer machine, or remotely on the build/test machines, in order to obtain a perfectly reproducible software development process that can be run anywhere without any difference and no surprises.

Reality is that (I can speak for Azure DevOps or GitHub workflows) server side workflows require Tasks or Actions, which are essentially something the developer cannot run locally on the development machine.

In short, the problem is the lack of 100% reproducible software development processes: the build/test process run by the developer on the local machine is not exactly identical to the one run on the server side workflow.

Let's create a reproducible workflow, not polluted with code the developer cannot run!

In this pure workflow, it is presented how to create a workflow that is as close as possible to what the developer would run locally on the development machine. The final intention is to give the developer the power to:

having a reproducible workflow based on well known tools (i.e. CMake, Ninja, vcpkg).
provide to the developer the knowledge to create a workflow without relying on any magic GitHub action.

The pure workflow is based on:

absence of unneeded GitHub actions.
the fact that vcpkg can be bootstrapped (i.e .building the vcpkg executable) when the vcpkg's toolchain file is passed in at CMake's project file generation time.
the vcpkg.json manifest file that specifies declaratively the dependencies to be installed.

Developers must own the knowledge and the choices

This information must be provided to the users in the readme.md files of run-vcpkg and run-cmake: users must own the knowledge and take a conscious decision about whether any GitHub action is necessary.

The future is even brighter

[x] CMake (already) supports a CMakePresets.json file to ease having a matching local and server side experience when configuring and building a project. This file is consumed directly by CMake when running, so it makes configuring a project exactly in the same way on the dev machine and on the build server machine a breeze.
[ ] according to its Roadmap, vcpkg will be distributed as a standalone binary (not yet done as of 4th Jan 2021) which would not require anymore to bootstrap it (i.e. building it). As of today the vcpkg tool executable can only install packages included in the same Git repository (i.e. same commit id) it was built from. To rephrase, the vcpkg tool and the data must exactly match and come from the same Git commit id.

Added values of a GitHub Action

There are still useful features provided by the actions that cannot be covered by tools, e.g.:

solving task automatically that the user does not have to think about. e.g. run-vcpkg automatically caches the artifacts. or get-cmake caches automatically Ninja and CMake.
providing error and warning matchers (aka problem matchers).

vadi2 commented 3 years ago

This toolchain comes from vcpkg, right?

lukka commented 3 years ago

@vadi2 yes! I added a link to the vcpkg toolchain file, and added more thoughts on this as well.

vadi2 commented 3 years ago

I definitely see how this will solve a problem and it looks really good.

That said something not mentioned is that in the real world not everything is available in vcpkg - and something you don't want to be in vcpkg, like header-only boost which vcpkg insists on building (15-20mins). So it would need to allow for custom scripts. See https://github.com/Mudlet/Mudlet/blob/development/.github/workflows/build-mudlet.yml#L80 where we have per-platform Install non-vcpkg dependencies steps.

Otherwise, it would be a dream to be able to use the same setup as CI on the desktop :+1:

vadi2 commented 3 years ago

How does https://github.com/TheLartians/CPM.cmake relate to this issue?

lukka commented 3 years ago

Looks like CPM.cmake is another way to install the dependencies by means of CMake itself, which is good as this would allow to have same workflow locally and on CI server. Interestingly it reminded me of another project very similar in nature and name, i.e. cpm.

@vadi2 if there any special feature or consideration about this project which would relate to this, let me know

vadi2 commented 3 years ago

I've checked out the roadmap but I don't see their plans for a standalone binary - at most, it'll be included inside Visual Studio/MSVC which we don't use on Windows - we use MingGW. We support Clang and GCC and dealing with two compilers is enough, MSVC is notorious for having incompatibilities with the other two. That adds a bit of a problem to these plans.

lukka commented 2 years ago

Created a project template that documents how to use a pure and a non-pure workflow: https://github.com/lukka/CppCMakeVcpkgTemplate This allows people to choose what fits their needs.

lukka / run-cmake