Publish the cross-compile toolchain as a container image

[qrkourier]

The purpose of this change is to improve the developer experience for T-SDK, specifically building the ziti-edge-tunnel (ZET) binary. Building the C-SDK and ZET packages (RPM, DEB) are not in scope for this issue.

The new container image will be versioned separately from T-SDK and live in a separate repo. It will be based on the unofficial cross-build container, which will be updated to use Ubuntu Bionic for GLIBC 2.27.

Then, a version of the new container image will be pinned to the CMake workflows for the T-SDK, replacing the current job-container, ubuntu:18.04 (link to workflow).

[qrkourier]

Please comment on whether this implementation meets your expectations for a cross-compile container image: https://github.com/openziti/ziti-tunnel-sdk-c-builder#readme.

[qrkourier]

This solution will replace the Debian-based cross-build container documented in this repo's README file.

[qrkourier]

@ekoby has suggested that we broaden the scope of the container image to be a more general-purpose build tool shared by the CI and local build scripts in the following projects:

• T-SDK
• C-SDK
• zitify

I am beginning to investigate the build requirements for C-SDK and zitify since they may be different from T-SDK. I propose the follow developer experience for local builds for each project:

A build.sh script in the repo will have the default behavior of emulating the CMake workflow in CI. Optionally, the developer may override the default build target, etc. by specifying arbitrary CMake options and arguments.

As a stretch goal, the build script could detect the host arch of the developer's workstation and automatically compile for that target arch. Please comment on whether this would meet your expectations.

Notably, I do not plan to design a CLI/API for the build procedure, i.e. replete opts and args for the developer to learn in order to interact with the build script. If this is an important goal for you then please comment on how you want that to work and why.

[qrkourier]

This solution will replace some of the steps in this repo's CMake CI workflow. For example, the steps that install and run cmake and vcpkg. If the same approach is adopted by the C-SDK or zitify repos then the same types of steps in those repos' CMake workflows will be replaced by a single step to run this container.

[ekoby]

I would prefer that Docker image does not make assumptions about the build process or scripts. For example in CI, I just want to use this image like this

jobs:
  build:
    runs-on: ubuntu-latest
    container:
      image: ziti-c-builder:latest

so I can still use github actions that provide useful utilities, like caching

I guess I just want an image that has all the tools -- cmake, git, gcc, gcc-crosscompile, etc

[qrkourier]

That makes sense. I'd like to propose the following solution summarizing the input I've received up to this point. Please comment on whether this meets your expectations for a CMake container image.

• a published builder container image openziti/ziti-cmake with WORKDIR /github/workspace for GH Actions compatibility
• image has all the necessary utilities and libraries for cross-compiling OpenZiti projects with CMake (those that employ the C-SDK)
• image does not run cmake or produce any particular artifact because it has no entrypoint script or default command except bash
• cmake.sh
  • each project will have a new shell script cmake.sh that is intended to be run inside the CMake container to build the default target for that project
  • cmake.sh accepts positional arguments that may be another script inside the project directory or positional commands to run instead of building the default target, e.g., ./cmake.sh run-my-tests.sh or ./cmake.sh --build ./build-arm --target bundle --verbose
  • cmake.sh will detect it is not running inside the ziti-cmake container image by probing for the existence of /github/workspace/cmake.sh and will re-run itself inside that container image with the project repo mounted on /github/workspace with --user $UID
  • cmake.sh will always delete /github/workspace/build before running cmake with build output dir ./build only when building the default target, not when running custom commands passed as arguments to cmake.sh
• the GitHub Actions CMake workflow for each project will use the ziti-cmake container as a job container in which to execute job steps, including a run step to run ./cmake.sh
• developers will run ./cmake.sh without arguments locally in each project directory when they wish to locally build the default target mirroring the GitHub Actions CMake workflow
• developers may enhance cmake.sh in each project
  • to optionally compile natively on their workstation instead of using the container, e.g., --no-docker
  • to optionally use an alternative container runtime/build cache, e.g. --podman or --nerdctl
  • to optionally use an alternative container image, e.g., --image IMAGE
  • with options and arguments to conveniently build different targets or skip tests

[qrkourier]

I have updated the README.md in this issue's resolving pull request to reflect the implementation sketch above.

[ekoby]

the prescription for cmake.sh seem superfluous in this context, the consumer project can do whatever, right?

[qrkourier]

the prescription for cmake.sh seem superfluous in this context, the consumer project can do whatever, right?

That's true. It's technically superfluous because the container image will not prescribe any command or use case, assuming the premeditated series of changes in the repositories that will consume this new container image are acceptable as described here. That assumption is probably correct. If that assumption is incorrect, and the consuming repos must use the image differently for some reason, then it will change the design of this image.

[qrkourier]

I've marked the associated pull request ready for review. After acceptance of the initial design, I plan to test the automatically published image with each of the Ziti projects that could adopt this image in their CI.

[qrkourier]

Here's an implementation of cmake.sh using this container image to cross-compile the Linux artifacts for the T-SDK in CI. A developer can run ./cmake.sh to build the default target for x86_64 locally, or ./cmake.sh -p ci-linux-arm64 to build the default target with an alternative os-architecture preset.

https://github.com/openziti/ziti-tunnel-sdk-c/pull/662

[qrkourier]

❯ ./cmake.sh help

Usage: cmake.sh [CMD] [ARGS...] 

Runs CMD in the ziti-cmake container, and builds the 
default target if no CMD is specified

    -c  CMAKE_CONFIG  set CMAKE_BUILD_TYPE (default: Release) 
    -p  CMAKE_PRESET  set CMAKE_TOOLCHAIN_FILE preset (default: ci-linux-x64) 
    -t  CMAKE_TARGET  set CMAKE_TARGET (default: bundle)

[qrkourier]

@scareything Will you react to the emulation experience on your M1 macOS? I've raised a new issue in this repo to improve the experience on arm64 silicon by running the build in an arm64 image instead of emulating amd64.