openstack-charmers / charmed-openstack-tester

Deploys OpenStack clouds and runs tempest (or other tests) against them
Apache License 2.0
6 stars 14 forks source link

Add tempest to charm-upgrade tests #45

Closed lourot closed 3 years ago

lourot commented 3 years ago

Validated locally:

$ tox -e charm-upgrade-focal-ussuri
[...]
======
Totals
======
Ran: 94 tests in 1143.3399 sec.
 - Passed: 85
 - Skipped: 9
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 0
Sum of execute time for each test: 484.1456 sec.

==============
Worker Balance
==============
 - Worker 0 (94 tests) => 0:19:03.339876
2021-07-18 22:05:47 [INFO] Events:
  Before Deploy zaza.openstack.configure.pre_deploy_certs.set_cidr_certs:
    Start: 1626637498.785418
    Finish: 1626637499.3062449
    Elapsed Time: 0.5208268165588379
    PCT Of Run Time: 1
  Configure zaza.openstack.charm_tests.ceilometer.setup.basic_setup:
    Start: 1626639733.0326116
    Finish: 1626639775.676689
    Elapsed Time: 42.64407730102539
    PCT Of Run Time: 1
  Configure zaza.openstack.charm_tests.glance.setup.add_cirros_alt_image:
    Start: 1626640983.3207092
    Finish: 1626641004.2218566
    Elapsed Time: 20.90114736557007
    PCT Of Run Time: 1
  Configure zaza.openstack.charm_tests.glance.setup.add_cirros_image:
    Start: 1626640961.3858614
    Finish: 1626640983.3206904
    Elapsed Time: 21.934828996658325
    PCT Of Run Time: 1
  Configure zaza.openstack.charm_tests.glance.setup.add_lts_image:
    Start: 1626641004.221886
    Finish: 1626641051.8833098
    Elapsed Time: 47.66142392158508
    PCT Of Run Time: 1
  Configure zaza.openstack.charm_tests.keystone.setup.add_demo_user:
    Start: 1626641233.145639
    Finish: 1626641292.9914203
    Elapsed Time: 59.845781326293945
    PCT Of Run Time: 1
  Configure zaza.openstack.charm_tests.keystone.setup.add_tempest_roles:
    Start: 1626641292.9914734
    Finish: 1626641308.6282196
    Elapsed Time: 15.636746168136597
    PCT Of Run Time: 1
  Configure zaza.openstack.charm_tests.neutron.setup.basic_overcloud_network:
    Start: 1626641051.8833494
    Finish: 1626641179.4472008
    Elapsed Time: 127.56385135650635
    PCT Of Run Time: 2
  Configure zaza.openstack.charm_tests.nova.setup.create_flavors:
    Start: 1626641179.4472604
    Finish: 1626641206.7423954
    Elapsed Time: 27.295135021209717
    PCT Of Run Time: 1
  Configure zaza.openstack.charm_tests.nova.setup.manage_ssh_key:
    Start: 1626641206.742456
    Finish: 1626641233.1455934
    Elapsed Time: 26.40313744544983
    PCT Of Run Time: 1
  Configure zaza.openstack.charm_tests.tempest.setup.render_tempest_config_keystone_v3:
    Start: 1626641308.6282575
    Finish: 1626641363.0488977
    Elapsed Time: 54.42064023017883
    PCT Of Run Time: 1
  Configure zaza.openstack.charm_tests.vault.setup.auto_initialize:
    Start: 1626639775.6767712
    Finish: 1626640961.3857744
    Elapsed Time: 1185.7090032100677
    PCT Of Run Time: 15
  Deploy Bundle:
    Start: 1626637499.306325
    Finish: 1626637676.251105
    Elapsed Time: 176.94478011131287
    PCT Of Run Time: 3
  Prepare Environment:
    Start: 1626637495.157025
    Finish: 1626637498.7851996
    Elapsed Time: 3.6281745433807373
    PCT Of Run Time: 1
  Test zaza.openstack.charm_tests.charm_upgrade.tests.FullCloudCharmUpgradeTest:
    Start: 1626642502.059527
    Finish: 1626644787.740485
    Elapsed Time: 2285.680958032608
    PCT Of Run Time: 28
  Test zaza.openstack.charm_tests.openstack_upgrade.tests.WaitForMySQL:
    Start: 1626641363.0490928
    Finish: 1626641364.1903005
    Elapsed Time: 1.1412076950073242
    PCT Of Run Time: 1
  Test zaza.openstack.charm_tests.tempest.tests.TempestTest:
    Start: 1626644787.7406707
    Finish: 1626645947.935788
    Elapsed Time: 1160.1951172351837
    PCT Of Run Time: 14
  Wait for Deployment:
    Start: 1626637676.2513938
    Finish: 1626639732.7069905
    Elapsed Time: 2056.4555966854095
    PCT Of Run Time: 25
Metadata: {}

_____________________________________________________________________________________________________ summary _____________________________________________________________________________________________________
  charm-upgrade-focal-ussuri: commands succeeded
  congratulations :)

Note that Zaza's summary is misleading, as TempestTest in fact really has been run successfully twice, as expected (once before and once after the charm upgrade).