search

sagemath / sage

Main repository of SageMath
https://www.sagemath.org
Other
1.36k stars 462 forks source link

Meta-ticket: Add Dockerfiles and CI scripts for integration testing of source and binary distributions and of downstream packages #29060

Open mkoeppe opened 4 years ago

mkoeppe commented 4 years ago

Testing of source and binary distributions relies too much on manual testing by people.

We propose to set up Dockerfiles and CI scripts that document and test the expected capabilities of source and binary Sage distributions; and of downstream distribution packaging.

Testing of source trees and source distributions: (in progress)

With the important changes brought by #27330 (Meta-ticket: spkg-configure: Try to use as many system packages as possible), sage-the-distribution has become less monolithic; it now interacts in a more complex way with distribution packaging. We propose to add infrastructure for testing correct installation of sage source distributions on a variety of platforms.

  1. We collect information about distribution packages systematically and store it in on a per-SPKG basis in build/pkgs/SPKG/distros/. (Right now this information is scattered - in downstream sage distribution packagers' build scripts, trac tickets, replies to bug reports in google groups, the sage installation manual, personal knowledge...) This is part of #29053, for debian/fedora/arch/conda (with follow-up tickets #29105, #29106 for other platforms).

  2. We generate well-defined test environments in the form of Dockerfiles. This makes it possible to test, on one's development computer, the correct installation of sage-the-distribution on a variety of platforms and configurations. #29053 provides build/bin/write-dockerfile.sh that generates the Dockerfile using the information in 1. The minimal configuration has just the packages that are needed for a build to succeed. The standard configuration installs all distribution packages that sage knows how to use.

  3. Running tests on many test environments is automated using tox in #29053. The top-level tox.ini file defines the test environments. Using tox's factor conditions (https://tox.readthedocs.io/en/latest/config.html#complex-factor-conditions), there is a succinct description of many test environments. For example, we can run tox -e docker-debian-stretch-minimal,docker-arch-latest-standard.

  4. 29087 automates running the tox tests for a selection of 34 platforms/configurations using a GitHub Actions workflow on every git push to a GitHub repository. An example run: https://github.com/mkoeppe/sage/actions/runs/32812271

Tickets:

Related meta-tickets:

Ideas without tickets so far:

Testing of the sage binary distributions: moved to #31133

Testing of downstream packaging of sage: (planning stage)

Symptoms:

CC: @vbraun @kiwifb @isuruf @dimpase @embray @saraedum @antonio-rojas @slel @sheerluck @tobiasdiez

Component: build

Keywords: ContinuousIntegration, sd111

Issue created by migration from https://trac.sagemath.org/ticket/29060

mkoeppe commented 2 years ago

Description changed:

--- 
+++ 
@@ -54,6 +54,7 @@
 - #30726: GH Actions: Test optional/experimental packages with Xcode 12
 - #29527: tox.ini: Add docker-opensuse
 - #30213: tox.ini: Add scientificlinux
+- #29356: Meta-ticket: more system pkgs info for various linuxes
 - #32113: GH Actions: `conda-forge` tests broken
 - #32947: `tox.ini`, GH Actions: Update Linux platforms
 - #33196: `tox.ini`: Add `centos-stream-8`, `centos-stream-9`, remove defunct `centos-6`
@@ -62,18 +63,19 @@
 - #31415: GH Actions workflow that builds documentation on each push
 - #33306: `tox.ini`: Linux platform updates
 - #32965: `tox` / GH Actions: Add `centos-7-devtoolset-gcc_{9,10,11}`
+- #32966: tox / GH Actions: Add ubuntu variants using `ppa:ubuntu-toolchain-r`
 - #33263: Add github action running on each push
-- #30845: GH Actions: Add test for conda without SPKG
+- #30845: tox.ini, GH Actions: Add test for conda without SPKG
 - #32703: GH Actions: Run a job for `make build-local` first, cache image for job `make build`
 - #31211: `ci-cygwin*.yml`: Fix packaging up of SAGE_LOCAL/var/tmp/sage/build/*
 - #30933: GH Actions: Repair upload of docker images
+- #31413: GH Actions: Add workflow with `configure --enable-editable`
+- #34228: `tox -e docker-...-incremental`
+- #33062: GH Actions (docker): Run a job for "make build-local" first, cache image for job "make build"
 - #32784: Filter singular system package out of `tox.ini` for homebrew 
 - #31088: GH Actions: WSL workflow: No space left on device
-- #31413: GH Actions: Add workflow with `configure --enable-editable`
 - #32570: `tox.ini`, GH Actions: Update macOS environments
-- #32966: tox / GH Actions: Add ubuntu variants using `ppa:ubuntu-toolchain-r`
 - #32531: `SAGE_ROOT/tox.ini`: Add variants that provision a system python using `pyenv`
-- #30845: tox.ini, GH Actions: Add test for conda without SPKG
 - #30505: tox.ini: Add `wsl` as a technology, to complement `local` and `docker`
 - #30924: tox.ini: Add `cygwin` as a technology, to complement `local` and `docker`
 - #29536: Make docker images from [GitHub](../wiki/GitHub) CI workflow and regular Sage Docker images interoperable
@@ -89,8 +91,8 @@
 - #33338 Refactor GH Actions workflows using https://docs.github.com/en/actions/learn-github-actions/reusing-workflows, also use reusable workflows for the upstream deployments of `ci-sage.yml` (suggested in https://github.com/pypa/setuptools/pull/2923#discussion_r795977169)

 Related meta-tickets:
-- #29356: Meta-ticket: more system pkgs info for various linuxes
 - #29146: Meta-ticket: Refactor/improve system package related scripts, `tox.ini`, `build/bin/write_dockerfile.sh`
+- #33338 Meta-ticket: CI workflows for upstream projects using Sage reusable workflows

 Ideas without tickets so far:
 - Testing that it is possible to build without error when upgrading with git from some list of previous releases.
mkoeppe commented 2 years ago

Description changed:

--- 
+++ 
@@ -93,6 +93,7 @@
 Related meta-tickets:
 - #29146: Meta-ticket: Refactor/improve system package related scripts, `tox.ini`, `build/bin/write_dockerfile.sh`
 - #33338 Meta-ticket: CI workflows for upstream projects using Sage reusable workflows
+- #32074 Meta-ticket: Remove support for outdated distros

 Ideas without tickets so far:
 - Testing that it is possible to build without error when upgrading with git from some list of previous releases.
mkoeppe commented 2 years ago

Description changed:

--- 
+++ 
@@ -108,7 +108,7 @@
 **Testing of downstream packaging of sage:** (planning stage)

 - #29111: Specify a subset of sage command line options that are supported by sagelib - rather than sage-the-distribution. Test that `sage -c SAGECOMMAND`, `sage -t`, `sage -python -c COMMAND`, ... work with the `sage` script provided by the distribution (see https://groups.google.com/d/msg/sage-packaging/BmkxIBdwbvE/fRMl2sjdBQAJ)
-
+- #33671 Add `devcontainer.json` for development with VS Code in a Docker container

 ---