search

openshift / ansible-service-broker

Ansible Service Broker
Apache License 2.0
226 stars 84 forks source link

[WIP][DO NOT MERGE] add Prow CI e2e test #1155

Closed jmontleon closed 5 years ago

openshift-ci-robot commented 5 years ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: jmontleon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/openshift/ansible-service-broker/blob/master/OWNERS)~~ [jmontleon] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
jmontleon commented 5 years ago

The way this works in theory is that we trigger the make task in Prow like all the other tests after bringing up a cluster using one of the templates provided. I've had success locally but haven't been able to test in Prow yet. It may also require merging to test.

jmontleon commented 5 years ago

/retest

jmontleon commented 5 years ago

/test images

jmontleon commented 5 years ago

/test images

coveralls commented 5 years ago

Coverage Status

Coverage remained the same at 40.97% when pulling 29d8c21745313f49d6e2cfa9c34d588f73109f0e on jmontleon:add-ocp-e2e into b54997cce43d410f9b8a226a27f0893b7a770909 on openshift:master.

jmontleon commented 5 years ago

Example run

$ make openshift-ci-e2e
git clone https://github.com/fusor/catbrokers4
Cloning into 'catbrokers4'...
remote: Enumerating objects: 41, done.
remote: Counting objects: 100% (41/41), done.
remote: Compressing objects: 100% (33/33), done.
remote: Total 41 (delta 10), reused 32 (delta 7), pack-reused 0
Unpacking objects: 100% (41/41), done.
pushd catbrokers4; ./run-playbook.sh; popd
~/Documents/openshift/src/github.com/openshift/ansible-service-broker/catbrokers4 ~/Documents/openshift/src/github.com/openshift/ansible-service-broker
 [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'

PLAY [localhost] ******************************************************************************************************************************************************************************

TASK [Gathering Facts] ************************************************************************************************************************************************************************
ok: [localhost]

TASK [Check if PackageManifests resource type can be retrieved] *******************************************************************************************************************************
changed: [localhost]

TASK [Delete packageserver subscription if PackageManifests aren't visible] *******************************************************************************************************************
skipping: [localhost]

TASK [Delete packageserver CSV if PackageManifests aren't visible] ****************************************************************************************************************************
skipping: [localhost]

TASK [Create packageserver subscription if PackageManifests aren't visible] *******************************************************************************************************************
skipping: [localhost]

TASK [Wait for packageserver pod to start running] ********************************************************************************************************************************************
skipping: [localhost]

TASK [Wait for PackageManifests resource to reappear once packageserver comes back] ***********************************************************************************************************
skipping: [localhost]

TASK [Scale down CVO so that limit removal will persist] **************************************************************************************************************************************
changed: [localhost]

TASK [Delete rh-operators ConfigMap] **********************************************************************************************************************************************************
changed: [localhost]

TASK [Delete rh-operators CatalogSource] ******************************************************************************************************************************************************
changed: [localhost]

TASK [Delete all pods in openshift-operator-lifecycle-manager namespace] **********************************************************************************************************************
changed: [localhost]

TASK [Wait for PackageManifests resource to reappear once packageserver comes back] ***********************************************************************************************************
changed: [localhost]

TASK [Wait for svcat PackageManifest to disappear] ********************************************************************************************************************************************
changed: [localhost]

TASK [Create rh-operators ConfigMap] **********************************************************************************************************************************************************
changed: [localhost]

TASK [Create rh-operators CatalogSource] ******************************************************************************************************************************************************
changed: [localhost]

TASK [Get catalog-operator pod name] **********************************************************************************************************************************************************
changed: [localhost]

TASK [Delete catalog-operator pod] ************************************************************************************************************************************************************
changed: [localhost]

TASK [Wait for svcat PackageManifest to appear] ***********************************************************************************************************************************************
changed: [localhost]

TASK [Create kube-service-catalog namespace] **************************************************************************************************************************************************
changed: [localhost]

TASK [Create kube-service-catalog OperatorGroup] **********************************************************************************************************************************************
changed: [localhost]

TASK [Delete existing kube-service-catalog Subscriptions] *************************************************************************************************************************************
changed: [localhost]

TASK [Delete existing kube-service-catalog CSVs] **********************************************************************************************************************************************
changed: [localhost]

TASK [Create kube-service-catalog Subscription] ***********************************************************************************************************************************************
changed: [localhost]

TASK [Wait for kube-service-catalog apiserver to become ready] ********************************************************************************************************************************
FAILED - RETRYING: Wait for kube-service-catalog apiserver to become ready (20 retries left).
FAILED - RETRYING: Wait for kube-service-catalog apiserver to become ready (19 retries left).
FAILED - RETRYING: Wait for kube-service-catalog apiserver to become ready (18 retries left).
FAILED - RETRYING: Wait for kube-service-catalog apiserver to become ready (17 retries left).
FAILED - RETRYING: Wait for kube-service-catalog apiserver to become ready (16 retries left).
FAILED - RETRYING: Wait for kube-service-catalog apiserver to become ready (15 retries left).
FAILED - RETRYING: Wait for kube-service-catalog apiserver to become ready (14 retries left).
FAILED - RETRYING: Wait for kube-service-catalog apiserver to become ready (13 retries left).
FAILED - RETRYING: Wait for kube-service-catalog apiserver to become ready (12 retries left).
changed: [localhost]

TASK [Wait for kube-service-catalog controller-manager to become ready] ***********************************************************************************************************************
FAILED - RETRYING: Wait for kube-service-catalog controller-manager to become ready (20 retries left).
FAILED - RETRYING: Wait for kube-service-catalog controller-manager to become ready (19 retries left).
FAILED - RETRYING: Wait for kube-service-catalog controller-manager to become ready (18 retries left).
FAILED - RETRYING: Wait for kube-service-catalog controller-manager to become ready (17 retries left).
FAILED - RETRYING: Wait for kube-service-catalog controller-manager to become ready (16 retries left).
FAILED - RETRYING: Wait for kube-service-catalog controller-manager to become ready (15 retries left).
changed: [localhost]

TASK [Create automation-broker namespace] *****************************************************************************************************************************************************
changed: [localhost]

TASK [Create automation-broker ConfigMap] *****************************************************************************************************************************************************
changed: [localhost]

TASK [Create automation-broker CatalogSource] *************************************************************************************************************************************************
changed: [localhost]

TASK [Create automation-broker OperatorGroup] *************************************************************************************************************************************************
changed: [localhost]

TASK [Wait for automationbroker PackageManifest to appear] ************************************************************************************************************************************
changed: [localhost]

TASK [Delete existing automation-broker Subscriptions] ****************************************************************************************************************************************
changed: [localhost]

TASK [Delete existing automation-broker CSVs] *************************************************************************************************************************************************
changed: [localhost]

TASK [Create automation-broker Subscription] **************************************************************************************************************************************************
changed: [localhost]

TASK [Wait for automationbroker resource type to exist] ***************************************************************************************************************************************
FAILED - RETRYING: Wait for automationbroker resource type to exist (10 retries left).
changed: [localhost]

TASK [Create automationbroker CR to create instance of broker] ********************************************************************************************************************************
changed: [localhost]

TASK [Wait for automation-broker-operator pod to start] ***************************************************************************************************************************************
FAILED - RETRYING: Wait for automation-broker-operator pod to start (30 retries left).
FAILED - RETRYING: Wait for automation-broker-operator pod to start (29 retries left).
FAILED - RETRYING: Wait for automation-broker-operator pod to start (28 retries left).
FAILED - RETRYING: Wait for automation-broker-operator pod to start (27 retries left).
FAILED - RETRYING: Wait for automation-broker-operator pod to start (26 retries left).
FAILED - RETRYING: Wait for automation-broker-operator pod to start (25 retries left).
FAILED - RETRYING: Wait for automation-broker-operator pod to start (24 retries left).
FAILED - RETRYING: Wait for automation-broker-operator pod to start (23 retries left).
changed: [localhost]

TASK [Wait for automation-broker pod to start] ************************************************************************************************************************************************
FAILED - RETRYING: Wait for automation-broker pod to start (20 retries left).
FAILED - RETRYING: Wait for automation-broker pod to start (19 retries left).
FAILED - RETRYING: Wait for automation-broker pod to start (18 retries left).
FAILED - RETRYING: Wait for automation-broker pod to start (17 retries left).
FAILED - RETRYING: Wait for automation-broker pod to start (16 retries left).
changed: [localhost]

TASK [Create automation-broker clusterrolebinding] ********************************************************************************************************************************************
changed: [localhost]

TASK [Create openshift-template-service-broker namespace] *************************************************************************************************************************************
changed: [localhost]

TASK [Create template-broker ConfigMap] *******************************************************************************************************************************************************
changed: [localhost]

TASK [Create template-broker CatalogSource] ***************************************************************************************************************************************************
changed: [localhost]

TASK [Create template-broker OperatorGroup] ***************************************************************************************************************************************************
changed: [localhost]

TASK [Wait for templateservicebroker PackageManifest to appear] *******************************************************************************************************************************
changed: [localhost]

TASK [Delete existing template-broker Subscriptions] ******************************************************************************************************************************************
changed: [localhost]

TASK [Delete existing template-broker CSVs] ***************************************************************************************************************************************************
changed: [localhost]

TASK [Create template-broker Subscription] ****************************************************************************************************************************************************
changed: [localhost]

TASK [Wait for templateservicebrokers resource type to exist] *********************************************************************************************************************************
FAILED - RETRYING: Wait for templateservicebrokers resource type to exist (10 retries left).
changed: [localhost]

TASK [Create templateservicebroker CR to create instance of broker] ***************************************************************************************************************************
changed: [localhost]

TASK [Wait for template-service-broker-operator pod to start] *********************************************************************************************************************************
FAILED - RETRYING: Wait for template-service-broker-operator pod to start (30 retries left).
FAILED - RETRYING: Wait for template-service-broker-operator pod to start (29 retries left).
changed: [localhost]

TASK [Wait for tsb apiserver-1 pod to start] **************************************************************************************************************************************************
FAILED - RETRYING: Wait for tsb apiserver-1 pod to start (20 retries left).
FAILED - RETRYING: Wait for tsb apiserver-1 pod to start (19 retries left).
FAILED - RETRYING: Wait for tsb apiserver-1 pod to start (18 retries left).
FAILED - RETRYING: Wait for tsb apiserver-1 pod to start (17 retries left).
FAILED - RETRYING: Wait for tsb apiserver-1 pod to start (16 retries left).
FAILED - RETRYING: Wait for tsb apiserver-1 pod to start (15 retries left).
changed: [localhost]

PLAY RECAP ************************************************************************************************************************************************************************************
localhost                  : ok=45   changed=44   unreachable=0    failed=0   

~/Documents/openshift/src/github.com/openshift/ansible-service-broker
rm -rf catbrokers4
echo -ne 'apiVersion: v1\nkind: PersistentVolume\nmetadata:\n  name: mw\nspec:\n  capacity:\n    storage: 1Gi\n  accessModes:\n    - ReadWriteOnce\n  hostPath:\n    path: /srv' >pv
oc create -f pv
persistentvolume/mw created
sleep 5m
oc create -f ansible_role/apb/test_manifest.yaml
namespace/automation-broker-apb created
serviceaccount/automation-broker-apb created
clusterrolebinding.rbac.authorization.k8s.io/automation-broker-apb created
pod/automation-broker-apb created
while true; do STAT=$(oc get po -n automation-broker-apb automation-broker-apb -o go-template='{{ .status.phase }}'); if [ "${STAT}" == "Running" ]; then break; fi; sleep 1; done
oc logs -f -n automation-broker-apb automation-broker-apb
DEPRECATED: APB playbooks should be stored at /opt/apb/project

PLAY [automation-broker-apb test playbook] *************************************

TASK [Provision test with CRDs] ************************************************
skipping: [localhost]

TASK [Create test namespace] ***************************************************
changed: [localhost]

TASK [Provision mediawiki] *****************************************************
changed: [localhost]

TASK [Wait for mediawiki service instance] *************************************
FAILED - RETRYING: Wait for mediawiki service instance (15 retries left).Result was: {
    "attempts": 1, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki service instance (14 retries left).Result was: {
    "attempts": 2, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki service instance (13 retries left).Result was: {
    "attempts": 3, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki service instance (12 retries left).Result was: {
    "attempts": 4, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki service instance (11 retries left).Result was: {
    "attempts": 5, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki service instance (10 retries left).Result was: {
    "attempts": 6, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki service instance (9 retries left).Result was: {
    "attempts": 7, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki service instance (8 retries left).Result was: {
    "attempts": 8, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki service instance (7 retries left).Result was: {
    "attempts": 9, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki service instance (6 retries left).Result was: {
    "attempts": 10, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki service instance (5 retries left).Result was: {
    "attempts": 11, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki service instance (4 retries left).Result was: {
    "attempts": 12, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki service instance (3 retries left).Result was: {
    "attempts": 13, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
ok: [localhost] => {
    "msg": "Instance ready status: [u'False']"
}

TASK [Provision postgresql] ****************************************************
changed: [localhost]

TASK [Wait for postgresql service instance] ************************************
FAILED - RETRYING: Wait for postgresql service instance (15 retries left).Result was: {
    "attempts": 1, 
    "changed": false, 
    "msg": "Instance ready status: []", 
    "retries": 16
}
FAILED - RETRYING: Wait for postgresql service instance (14 retries left).Result was: {
    "attempts": 2, 
    "changed": false, 
    "msg": "Instance ready status: []", 
    "retries": 16
}
FAILED - RETRYING: Wait for postgresql service instance (13 retries left).Result was: {
    "attempts": 3, 
    "changed": false, 
    "msg": "Instance ready status: []", 
    "retries": 16
}
FAILED - RETRYING: Wait for postgresql service instance (12 retries left).Result was: {
    "attempts": 4, 
    "changed": false, 
    "msg": "Instance ready status: []", 
    "retries": 16
}
FAILED - RETRYING: Wait for postgresql service instance (11 retries left).Result was: {
    "attempts": 5, 
    "changed": false, 
    "msg": "Instance ready status: []", 
    "retries": 16
}
FAILED - RETRYING: Wait for postgresql service instance (10 retries left).Result was: {
    "attempts": 6, 
    "changed": false, 
    "msg": "Instance ready status: []", 
    "retries": 16
}
FAILED - RETRYING: Wait for postgresql service instance (9 retries left).Result was: {
    "attempts": 7, 
    "changed": false, 
    "msg": "Instance ready status: []", 
    "retries": 16
}
ok: [localhost] => {
    "msg": "Instance ready status: []"
}

TASK [Create binding] **********************************************************
changed: [localhost]

TASK [Wait for mediawiki-postgresql binding instance] **************************
ok: [localhost] => {
    "msg": "Binding ready status: [u'False']"
}

TASK [Update mediawiki deployment(config)] *************************************
changed: [localhost]

TASK [Wait for mediawiki deployment(config) to be updated] *********************
FAILED - RETRYING: Wait for mediawiki deployment(config) to be updated (15 retries left).Result was: {
    "attempts": 1, 
    "changed": false, 
    "msg": "Waiting for mediawiki", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki deployment(config) to be updated (14 retries left).Result was: {
    "attempts": 2, 
    "changed": false, 
    "msg": "Waiting for mediawiki", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki deployment(config) to be updated (13 retries left).Result was: {
    "attempts": 3, 
    "changed": false, 
    "msg": "Waiting for mediawiki", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki deployment(config) to be updated (12 retries left).Result was: {
    "attempts": 4, 
    "changed": false, 
    "msg": "Waiting for mediawiki", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki deployment(config) to be updated (11 retries left).Result was: {
    "attempts": 5, 
    "changed": false, 
    "msg": "Waiting for mediawiki", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki deployment(config) to be updated (10 retries left).Result was: {
    "attempts": 6, 
    "changed": false, 
    "msg": "Waiting for mediawiki", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki deployment(config) to be updated (9 retries left).Result was: {
    "attempts": 7, 
    "changed": false, 
    "msg": "Waiting for mediawiki", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki deployment(config) to be updated (8 retries left).Result was: {
    "attempts": 8, 
    "changed": false, 
    "msg": "Waiting for mediawiki", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki deployment(config) to be updated (7 retries left).Result was: {
    "attempts": 9, 
    "changed": false, 
    "msg": "Waiting for mediawiki", 
    "retries": 16
}
ok: [localhost] => {
    "msg": "Waiting for mediawiki"
}

TASK [Wait for mediawiki deployment(config) to be available] *******************
ok: [localhost] => {
    "msg": [
        "Deployment(config) ready status: [u'True']"
    ]
}

TASK [Verify mediawiki] ********************************************************
ok: [localhost]

TASK [Delete binding] **********************************************************
changed: [localhost]

TASK [Wait for mediawiki-postgresql binding instance to be removed] ************
FAILED - RETRYING: Wait for mediawiki-postgresql binding instance to be removed (15 retries left).Result was: {
    "attempts": 1, 
    "changed": false, 
    "msg": "Binding ready status: [u'False']", 
    "retries": 16
}
ok: [localhost] => {
    "msg": "Binding ready status: [u'False']"
}

TASK [Deprovision postgresql] **************************************************
changed: [localhost]

TASK [Wait for postgresql service instance to be removed] **********************
FAILED - RETRYING: Wait for postgresql service instance to be removed (15 retries left).Result was: {
    "attempts": 1, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
FAILED - RETRYING: Wait for postgresql service instance to be removed (14 retries left).Result was: {
    "attempts": 2, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
FAILED - RETRYING: Wait for postgresql service instance to be removed (13 retries left).Result was: {
    "attempts": 3, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
FAILED - RETRYING: Wait for postgresql service instance to be removed (12 retries left).Result was: {
    "attempts": 4, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
ok: [localhost] => {
    "msg": "Instance ready status: [u'False']"
}

TASK [Deprovision mediawiki] ***************************************************
changed: [localhost]

TASK [Wait for mediawiki service instance to be removed] ***********************
FAILED - RETRYING: Wait for mediawiki service instance to be removed (15 retries left).Result was: {
    "attempts": 1, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki service instance to be removed (14 retries left).Result was: {
    "attempts": 2, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki service instance to be removed (13 retries left).Result was: {
    "attempts": 3, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
FAILED - RETRYING: Wait for mediawiki service instance to be removed (12 retries left).Result was: {
    "attempts": 4, 
    "changed": false, 
    "msg": "Instance ready status: [u'False']", 
    "retries": 16
}
ok: [localhost] => {
    "msg": "Instance ready status: [u'False']"
}

TASK [Delete test namespace] ***************************************************
changed: [localhost]

TASK [Wait for test namespace to be destroyed] *********************************
FAILED - RETRYING: Wait for test namespace to be destroyed (10 retries left).
FAILED - RETRYING: Wait for test namespace to be destroyed (9 retries left).
FAILED - RETRYING: Wait for test namespace to be destroyed (8 retries left).
FAILED - RETRYING: Wait for test namespace to be destroyed (7 retries left).
FAILED - RETRYING: Wait for test namespace to be destroyed (6 retries left).
ok: [localhost]

TASK [Deprovision test with CRDs] **********************************************
skipping: [localhost]

TASK [Wait for namespace to be destroyed] **************************************
skipping: [localhost]

PLAY RECAP *********************************************************************
localhost                  : ok=19   changed=9    unreachable=0    failed=0    skipped=3   

EXIT_CODE=$(oc get pod -n automation-broker-apb automation-broker-apb -o go-template="{{ range .status.containerStatuses }}{{.state.terminated.exitCode}}{{ end }}")
exit ${EXIT_CODE}
djzager commented 5 years ago

Nice work. I'm going to avoid putting any of the lifecycle stuff/labels on this until it's ready but I appreciated the example run.

openshift-ci-robot commented 5 years ago

@jmontleon: PR needs rebase.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
openshift-ci-robot commented 5 years ago

@jmontleon: The following tests failed, say /retest to rerun them all:

Test name Commit Details Rerun command
ci/prow/images 29d8c21745313f49d6e2cfa9c34d588f73109f0e link /test images
ci/prow/operator-molecule-e2e 29d8c21745313f49d6e2cfa9c34d588f73109f0e link /test operator-molecule-e2e

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).