Automated Upgrade Test

[omaciel]

The objective is to take a fully "populated" system and upgrade it to whatever is the latest available version.

Make sure you have created:

• provider
• product
• repo
• environment
• changeset
• template

Repo for example can be created from: http://inecas.fedorapeople.org/fakerepos/zoo3/

• Sync the repo,.
• Add repo to template.
• Add the product and template to changeset and promote it.

Subscribe a system by CLI commands:

• subscription-manager register --username admin --password admin --org $orgname --environment $envname --name $systemname
• get the poolID (Id field of result) from katello by CLI: katello -uadmin -padmin org subscriptions --name $orgname
• subscription-manager subscribe --pool $poolID

The upgrade should be done based on https://fedorahosted.org/katello/wiki/Upgrade and once completed, we should check that the following items are still enabled and accessible:

• provider
• product
• repo
• environment
• changeset
• template
• systems

[weissjeffm]

Some notes (mostly to myself for later):

I would expect that the bulk of "data dependent" upgrade testing should be done via the CLI/API. That is the tests where you put data into katello, upgrade, and then test that the data you put in still "works".

The post-upgrade UI testing would probably mostly be the same tests as were run before the upgrade (not necessarily operating on the same data, but a few of these tests would be nice). We could have UI tests that look for say an org whose name starts with "preupgrade" and if it's there, test it, if not, assume it's not an upgraded server and skip it. Obviously this would give a false negative if upgrade was trashing orgs, but this test should be covered on the CLI side.