search

doctrine / dbal

Doctrine Database Abstraction Layer
https://www.doctrine-project.org/projects/dbal.html
MIT License
9.49k stars 1.34k forks source link

Building the CI Build Matrix #6620

Open morozov opened 1 week ago

morozov commented 1 week ago

The goal is to provide high test coverage of the library code together with various supported dependencies while keeping the number of test jobs to the necessary minimum.

Types of Dependencies

  1. PHP versions: Every supported minor version and nightly.
  2. Database platforms and their versions (platform-specific, see below).
  3. Composer packages: Lowest and highest versions.

Database Platform Versions to be Tested

  1. The lowest and highest supported versions.
  2. Intermediate versions for which there is a dedicated code path in the library.
  3. LTS releases are preferred over non-LTS ones, even if the dedicated code path would be triggered by an older non-LTS version.

Proposed Pattern

  1. Latest stable PHP version: Test all combinations with each supported driver and all respective database platform versions matching the above requirements.
  2. Other supported PHP versions: Test with each supported driver and the latest version of the respective database platform.
  3. SQLite drivers: Test with the lowest supported and latest stable PHP versions.
  4. Composer dependencies:
    • Use the lowest Composer dependencies with the lowest supported PHP version and pdo_sqlite (our historical default driver).
    • Use the highest Composer dependencies in all other builds.

Code Coverage

Code coverage can serve as an indicator of over- or under-testing. If the removal of a job that uses some intermediate dependency version(s) does not result in a drop in code coverage, it means that no dedicated code path exists for this configuration. In such a case, it is a strong signal of that testing this configuration is unnecessary, and thus, we don't want to spend machine time testing it.

morozov commented 5 days ago

As of https://github.com/doctrine/dbal/pull/6624, we want to test only the lowest, highest and nightly PHP versions. Additionally, we may test the intermediate PHP versions if there's a dedicated code path used to support them (to be determined by code coverage).