Create default data for upgrade tests

[cswiii]

The following are good examples of things to populate in a pre-upgrade scenario. It is not exhaustive perhaps, but a good start. Feel free to expound.

• Orgs
• Users
• Manifests
• Content
  • RH
  • Custom
  • Puppet
  • Container
• Environments
• Content Views
  • Staged? (Not yet published)
  • Published
  • Promoted
• Activation Keys ...
• Provisioned Hosts
• Registered Hosts with consumed subscriptions/content
• P2 Capsule(s) synced with content

Populated content should(?) preferably be populated with easily-readable text, the rationale being that there will probably -- at least in the beginning -- be some manual examination of the populated data, and it will be easier to compare side by side. If we want to write some script though, later, that extracts the contents of each system and compares them for us, great!

"""
@Feature: Upgrade Tests
@Test: Server: Upgrade completes successfully.
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  Attempt to upgrade to latest version/compose of product; observe logs while/after doing so
@Assert: Upgrade is successful; all services start; logs show no ERRORs upon install or restart
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: Server: Upgraded Satellite loses no data upon upgrade.
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  Via $method (manual examination, automated comparison, db forensics/health-check), assure that all populated data remains exposed to user; any newly opened ports that might exist on upgraded instance do in fact get opened on an upgrade.
@Assert: No lost and/or irretrivable data.
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: Server:  Delta functionality between old version and latest release/compose works in upgraded system
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  Attempt to perform functionality critical to latest release in an updated system
@Assert: All new functionality is available and works as expected.
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: Server: Perform API Automation suite (dirty system)
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  Execute API automation suite against updated system
@Assert: No new or unexpected issues occur when executing API suite against upgraded server.  Note that latest version of API tests must be run, as API might change between versions
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: Server: Perform CLI Automation suite (dirty system)
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  Execute API automation suite against updated system
@Assert: No new or unexpected issues occur when executing CLI suite against upgraded server.  Note that latest version of CLI tests must be run, as CLI might change between versions
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: Server: Perform GUI Automation suite (dirty system)
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  Execute GUI automation suite against updated system
@Assert: No new or unexpected issues occur when executing GUI suite against upgraded server.  Note that latest version of GUI tests must be run, as GUI might change between versions
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: Server: Perform API Automation suite (clean system)
@Setup: Install a fresh, older release and immediately upgrade it
 1.  Execute API automation suite against updated system
@Assert: No new or unexpected issues occur when executing API suite against upgraded server.  Note that latest version of API tests must be run, as API might change between versions
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: Server: Perform CLI Automation suite (clean system)
@Setup: Install a fresh, older release and immediately upgrade it
 1.  Execute API automation suite against updated system
@Assert: No new or unexpected issues occur when executing CLI suite against upgraded server.  Note that latest version of CLI tests must be run, as CLI might change between versions
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: Server: Perform GUI Automation suite (clean system)
@Setup: Install a fresh, older release and immediately upgrade it
 1.  Execute GUI automation suite against updated system
@Assert: No new or unexpected issues occur when executing GUI suite against upgraded server.  Note that latest version of GUI tests must be run, as GUI might change between versions
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: Capsule: Upgrade completes successfully.
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  Attempt to upgrade to latest version/compose of product; observe logs while/after doing so
@Assert: Upgrade is successful; all services start; logs show no ERRORs upon install or restart
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: Capsule: Upgrade can communicate with upgraded server.
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  Attempt to sync content from upgraded host
@Assert: Functionality remains in upgraded capsule when connecting to upgraded server.
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: Capsule: Old capsule communicating with upgraded server
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  Attempt to sync content/communicate with upgraded host
@Assert: **UNSURE** At a minimum, capsule handles situation gracefully and does not cause disruption on parent server using older functionality; newer, unavailable functionality does not trigger issues in capsule.
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: katello-agent: Upgrade completes successfully.
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  Attempt to upgrade to latest version/compose of product; observe logs while/after doing so
@Assert: Upgrade is successful; all services start; logs show no ERRORs upon install or restart
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: katello-agent: Upgrade can communicate with upgraded server.
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  Attempt to communicate with upgraded host
@Assert: Functionality remains in upgraded capsule when connecting to upgraded server; system remains "registered" to upgraded host and can (??) consume content
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: katello-agent: Old katello-agent communicating with upgraded server
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  Attempt to sync content/communicate with upgraded host
@Assert: **UNSURE** At a minimum, agent handles situation gracefully and does not cause disruption on parent server using older functionality; newer, unavailable functionality does not trigger issues in agent.
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: katello-agent: Old katello-agent communicating with upgraded server
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  Attempt to sync content/communicate with upgraded host
@Assert: **UNSURE** katello-agent should be notified of new version and upgraded on the fly?
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: puppet-agent: Upgrade completes successfully.
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  Attempt to upgrade to latest version/compose of product; observe logs while/after doing so
@Assert: Upgrade is successful; all services start; logs show no ERRORs upon install or restart
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: puppet-agent: Upgrade can communicate with upgraded server.
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  Attempt to communicate with upgraded host
@Assert: Functionality remains in upgraded capsule when connecting to upgraded server.
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: puppet-agent: Old puppet-agent communicating with upgraded server
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  Attempt to sync content/communicate with upgraded host
@Assert: **UNSURE** At a minimum, agent handles situation gracefully and does not cause disruption on parent server using older functionality; newer, unavailable functionality does not trigger issues in agent.
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: puppet-agent: Old puppet-agent communicating with upgraded server
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  Attempt to sync content/communicate with upgraded host
@Assert: **UNSURE** puppet-agent should be notified of new version and upgraded on the fly?
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: katello-disconnected: Old katello-disconnected content is consumable by upgraded server
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  On older version of katello-disconnected, create/sync content
 2. Upon upgrading sat instance, attempt to sync this content
@Assert: **UNSURE** Content created via older katello-disconnected method can still be imported in newer sat.
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: rhsm: Previously registered systems remain registered to upgraded instance
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  On older version of satellite, before upgrading, register system(s)
 2. Upon upgrading, attempt to subscribe to new content via rhsm
@Assert: System remains registered
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: rhsm: Previously registered systems remain registered to upgraded instance
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1.  On older version of satellite, before upgrading, register system(s)
 2. Upon upgrading, attempt to subscribe to new content via rhsm
@Assert: System remains registered
@Status: Manual
"""
"""
@Feature: Upgrade Tests
@Test: rhsm: Systems can be registered to newly upgraded server
@Setup: Follow procedures to populate a previous GA version with selected content; upgrade this system.
 1. After upgrading, attempt to register a new client to server.
 @Assert: System registers successfully.
@Status: Manual
"""

[rochacbruno]

Also related to #4004

[ntkathole]

This is part of satellite6-upgrade project. We have existence tests. Please feel free to re-open, if this still needs fixing.