search

openfisca / openfisca-core

OpenFisca core engine. See other repositories for countries-specific code & data.
https://openfisca.org
GNU Affero General Public License v3.0
168 stars 75 forks source link

Simplify testing of new major versions #1158

Closed MattiSG closed 1 year ago

MattiSG commented 1 year ago

For smoke testing purposes, the Core has a dependency on both Country Template and Extension Template. This enables running their test suites with the latest Core code.

While powerful and useful, this technique leads to a major obstacle in delivering major releases of the Core. Indeed, both of these packages in turn have a dependency to the core, and this dependency is pinned to a specific major version to ensure stability.

These two points combined create a circular dependency that prevents testing major releases: if I bump the major Core version to version N, the Template dependencies bring irresolvable constraints to version N-1.

This is typically handled by creating a new branch for each repository, relaxing the version constraint, and updating the core dependency to these new branches. This means that every such change needs temporary modifications on at least three repositories, making them very impractical. See for example https://github.com/openfisca/openfisca-core/pull/989, leading to https://github.com/openfisca/country-template/pull/128 and https://github.com/openfisca/extension-template/pull/47.

A solution to enable testing from the Core with any Core version would significantly ease the delivery of new major versions, which will be needed to finally unblock many PRs and features that have been waiting for way too long.

MattiSG commented 1 year ago

Suggestion 1

  1. Delay major version bump in Core until PR has been approved, which means a code freeze.
  2. Upgrade higher bound in template repositories, without upgrading lower bound, and publish new version.
  3. Bump major in Core, merge, publish.

Example

  1. Approve https://github.com/openfisca/openfisca-core/pull/989.
  2. In https://github.com/openfisca/country-template/pull/128, upgrade Core constraint from OpenFisca-Core[web-api] >= 35.0.0, < 36.0.0 to OpenFisca-Core[web-api] >= 35.0.0, < 37.0.0, publish OpenFisca-Country-Template@4.
  3. In https://github.com/openfisca/extension-template/pull/47, upgrade Core constraint from OpenFisca-Country-Template >= 3.8.0, < 4 to OpenFisca-Country-Template >= 3.8.0, < 5, publish OpenFisca-Extension-Template@2.
  4. Bump major in https://github.com/openfisca/openfisca-core/pull/989. Tests should pass. Merge and publish.

Suggestion 2

Add a dependency group in Country Template and Extension Template called something like future that removes the upper bound version constraint on the Core, making it easier to test upgrades by changing the dependency in Core from openfisca-country-template to openfisca-country-template[future].

Example

  1. Add an optional dependency group in Country Template and Extension Template called future (or testing, or optimistic, or latest-core) that overrides the dependency to OpenFisca-Core[web-api] >= 35.0.0, < 36.0.0 to OpenFisca-Core[web-api] >= 35.0.0.
  2. Change Core dependency from OpenFisca-Country-Template to OpenFisca-Country-Template[future].
  3. Have all tests pass in Core without making any effort. If there are errors, then that means code upgrades in Template are actually warranted.
  4. Publish Core update.
  5. In https://github.com/openfisca/country-template/pull/128, upgrade Core constraint from OpenFisca-Core[web-api] >= 35.0.0, < 36.0.0 to OpenFisca-Core[web-api] >= 35.0.0, < 37.0.0, publish OpenFisca-Country-Template@4.
benjello commented 1 year ago

I like suggestion 2 as it imposes less constraints at each step of the process and report those efforts to where knowledge can be found (I use core but do not use the web-api for example).

MattiSG commented 1 year ago

Suggestion 3

Remove constraint on packages (#1160)