search

kinvolk / lokomotive

🪦 DISCONTINUED Further Lokomotive development has been discontinued. Lokomotive is a 100% open-source, easy to use and secure Kubernetes distribution from the volks at Kinvolk
https://kinvolk.io/lokomotive-kubernetes/
Apache License 2.0
320 stars 49 forks source link

Run upgrade tests from master to PR code as part of CI process #12

Open invidian opened 4 years ago

invidian commented 4 years ago

To ensure, that our users can update to the next version of Lokomotive, we should test upgrade process via CI, the same way we test fresh installations.

BrainBlasted commented 4 years ago

Tested our current status on Lokomotive upgrades, and it seems like everything is in good shape. For CI like this, I'm not sure where to get started.

invidian commented 4 years ago

@BrainBlasted I'd try doing something like this on the test CI pipeline:

  1. Checkout latest release tag.
  2. Create cluster from CI configuration.
  3. Checkout master branch.
  4. Run lokoctl cluster apply
  5. Run e2e tests to make sure everything is still operational.
iaguis commented 4 years ago

I have some ugly WIP code for this:

https://github.com/kinvolk/lokomotive/tree/iaguis/upgrades-test https://github.com/kinvolk/lokomotive-ci/tree/iaguis/upgrades

I added a specific configuration for a tagged release (v0.2.1) since it might change between the last release and master.

iaguis commented 4 years ago

@johananl suggested adding a generic upgrade.sh script that we can call on the CI and can also be useful for the user.

@invidian suggested fully handling upgrades in lokoctl, but we might want to do the script path first to have something working.

johananl commented 4 years ago

@johananl suggested adding a generic upgrade.sh script that we can call on the CI and can also be useful for the user.

To elaborate, my thought was to have a per-release upgrade script which includes any manual operations required for upgrading from version N-1 to version N. This script can use some naming convention which the CI can use to generically invoke upgrade scripts, and users can learn about the script from the release notes.

BrainBlasted commented 4 years ago

By "per-release upgrade script", do you mean having multiple upgrade scripts within the source for each lokomotive version?

johananl commented 4 years ago

By "per-release upgrade script", do you mean having multiple upgrade scripts within the source for each lokomotive version?

I haven't thought about the low-level details too much, my point was that IMO it could make sense to ensure that every Lokomotive release include the logic for upgrading to that release as part of the release. Such logic could then be used both by users and by our CI.

invidian commented 4 years ago

Note: I think with #312 implemented this issue would be easier to address.

surajssd commented 4 years ago

This issue says that we should run the upgrade test as a part of CI process. Does that mean we should have this in a separate pipeline?

invidian commented 4 years ago

This issue says that we should run the upgrade test as a part of CI process. Does that mean we should have this in a separate pipeline?

Yes, which might be triggered on demand.