search

openstack-k8s-operators / edpm-ansible

External Dataplane Management Ansible Playbooks
https://openstack-k8s-operators.github.io/edpm-ansible/
Apache License 2.0
9 stars 65 forks source link

Add data-plane-adoption job to the edpm-ansible zuul github-check layout #662

Closed marios closed 3 months ago

marios commented 3 months ago

This adds cifmw-data-plane-adoption-osp-17-to-extracted-crc to github-check.

https://issues.redhat.com/browse/OSPRH-7332

openshift-ci[bot] commented 3 months ago

Skipping CI for Draft Pull Request. If you want CI signal for your change, please convert it to an actual PR. You can still manually trigger a test run with /test all

softwarefactory-project-zuul[bot] commented 3 months ago

Build failed (check pipeline). Post recheck (without leading slash) to rerun all jobs. Make sure the failure cause has been resolved before you rerun jobs.

https://review.rdoproject.org/zuul/buildset/1dfdefb19ab34d44bad29b33b34baead

:heavy_check_mark: openstack-k8s-operators-content-provider SUCCESS in 4h 24m 33s :x: podified-multinode-edpm-deployment-crc FAILURE in 2h 18m 35s :heavy_check_mark: cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 47m 35s :heavy_check_mark: cifmw-data-plane-adoption-osp-17-to-extracted-crc SUCCESS in 3h 06m 59s :heavy_check_mark: cifmw-data-plane-adoption-osp-17-to-extracted-crc-minimal-no-ceph SUCCESS in 3h 01m 59s :heavy_check_mark: edpm-ansible-molecule-edpm_bootstrap SUCCESS in 6m 33s :heavy_check_mark: edpm-ansible-molecule-edpm_podman SUCCESS in 5m 20s :heavy_check_mark: edpm-ansible-molecule-edpm_module_load SUCCESS in 4m 45s :heavy_check_mark: edpm-ansible-molecule-edpm_kernel SUCCESS in 10m 22s :heavy_check_mark: edpm-ansible-molecule-edpm_libvirt SUCCESS in 8m 13s :heavy_check_mark: edpm-ansible-molecule-edpm_nova SUCCESS in 8m 35s :heavy_check_mark: edpm-ansible-molecule-edpm_frr SUCCESS in 6m 41s :heavy_check_mark: edpm-ansible-molecule-edpm_iscsid SUCCESS in 4m 43s :heavy_check_mark: edpm-ansible-molecule-edpm_ovn_bgp_agent SUCCESS in 6m 56s :heavy_check_mark: edpm-ansible-molecule-edpm_ovs SUCCESS in 5m 21s :heavy_check_mark: edpm-ansible-molecule-edpm_tripleo_cleanup SUCCESS in 4m 08s

marios commented 3 months ago

I have the same comment here as I do on openstack-k8s-operators/dataplane-operator#902

openstack-k8s-operators/dataplane-operator#902 (review)

ack same as https://github.com/openstack-k8s-operators/dataplane-operator/pull/902#pullrequestreview-2087757154

let's add one job here explicitly and we can target it with files: for some of the roles we are most interested in for example

softwarefactory-project-zuul[bot] commented 3 months ago

Build failed (check pipeline). Post recheck (without leading slash) to rerun all jobs. Make sure the failure cause has been resolved before you rerun jobs.

https://review.rdoproject.org/zuul/buildset/74d8305caaed4ab1ad5330c11259aae0

:heavy_check_mark: openstack-k8s-operators-content-provider SUCCESS in 3h 41m 23s :x: podified-multinode-edpm-deployment-crc FAILURE in 2h 17m 42s :heavy_check_mark: cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 44m 23s :heavy_check_mark: edpm-ansible-molecule-edpm_bootstrap SUCCESS in 7m 10s :heavy_check_mark: edpm-ansible-molecule-edpm_podman SUCCESS in 5m 26s :heavy_check_mark: edpm-ansible-molecule-edpm_module_load SUCCESS in 4m 44s :heavy_check_mark: edpm-ansible-molecule-edpm_kernel SUCCESS in 9m 05s :heavy_check_mark: edpm-ansible-molecule-edpm_libvirt SUCCESS in 8m 12s :heavy_check_mark: edpm-ansible-molecule-edpm_nova SUCCESS in 8m 39s :heavy_check_mark: edpm-ansible-molecule-edpm_frr SUCCESS in 6m 07s :heavy_check_mark: edpm-ansible-molecule-edpm_iscsid SUCCESS in 4m 37s :heavy_check_mark: edpm-ansible-molecule-edpm_ovn_bgp_agent SUCCESS in 6m 34s :heavy_check_mark: edpm-ansible-molecule-edpm_ovs SUCCESS in 5m 13s :heavy_check_mark: edpm-ansible-molecule-edpm_tripleo_cleanup SUCCESS in 3m 53s :heavy_check_mark: cifmw-data-plane-adoption-osp-17-to-extracted-crc SUCCESS in 3h 12m 15s

marios commented 3 months ago

check-rdo

marios commented 3 months ago

I think that if we are to run this at all it must be very selective. Furthermore, we must make sure that we aren't using this to test things which are already covered by molecule tests or other jobs.

Is there a role which absolutely requires this test?

there is a related conversation ongoing at https://github.com/openstack-k8s-operators/dataplane-operator/pull/902#discussion_r1624050848 FYI

wrt to your question, this is the list of services we are deploying https://github.com/marios/data-plane-adoption/blob/ec0f395a8321dc4c3ced71b777102ad7dc9de1b2/tests/roles/dataplane_adoption/tasks/main.yaml#L256 (corresponding to the plays and roles from this repo being executed )

ciecierski commented 3 months ago

I think that if we are to run this at all it must be very selective. Furthermore, we must make sure that we aren't using this to test things which are already covered by molecule tests or other jobs. Is there a role which absolutely requires this test?

there is a related conversation ongoing at openstack-k8s-operators/dataplane-operator#902 (comment) FYI

wrt to your question, this is the list of services we are deploying https://github.com/marios/data-plane-adoption/blob/ec0f395a8321dc4c3ced71b777102ad7dc9de1b2/tests/roles/dataplane_adoption/tasks/main.yaml#L256 (corresponding to the plays and roles from this repo being executed )

On top of that we have very specific adoption role: https://github.com/openstack-k8s-operators/edpm-ansible/tree/main/roles/edpm_pre_adoption_validation

softwarefactory-project-zuul[bot] commented 3 months ago

Build failed (check pipeline). Post recheck (without leading slash) to rerun all jobs. Make sure the failure cause has been resolved before you rerun jobs.

https://review.rdoproject.org/zuul/buildset/396f29b4806c48b68decc7562de4cb13

:heavy_check_mark: openstack-k8s-operators-content-provider SUCCESS in 3h 28m 19s :x: podified-multinode-edpm-deployment-crc FAILURE in 2h 16m 23s :heavy_check_mark: cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 48m 27s :heavy_check_mark: edpm-ansible-molecule-edpm_bootstrap SUCCESS in 6m 04s :heavy_check_mark: edpm-ansible-molecule-edpm_podman SUCCESS in 5m 14s :heavy_check_mark: edpm-ansible-molecule-edpm_module_load SUCCESS in 4m 36s :heavy_check_mark: edpm-ansible-molecule-edpm_kernel SUCCESS in 9m 26s :heavy_check_mark: edpm-ansible-molecule-edpm_libvirt SUCCESS in 8m 19s :heavy_check_mark: edpm-ansible-molecule-edpm_nova SUCCESS in 8m 35s :heavy_check_mark: edpm-ansible-molecule-edpm_frr SUCCESS in 6m 22s :heavy_check_mark: edpm-ansible-molecule-edpm_iscsid SUCCESS in 4m 35s :heavy_check_mark: edpm-ansible-molecule-edpm_ovn_bgp_agent SUCCESS in 6m 56s :heavy_check_mark: edpm-ansible-molecule-edpm_ovs SUCCESS in 4m 59s :heavy_check_mark: edpm-ansible-molecule-edpm_tripleo_cleanup SUCCESS in 4m 02s :heavy_check_mark: cifmw-data-plane-adoption-osp-17-to-extracted-crc SUCCESS in 3h 11m 53s

softwarefactory-project-zuul[bot] commented 3 months ago

Build failed (check pipeline). Post recheck (without leading slash) to rerun all jobs. Make sure the failure cause has been resolved before you rerun jobs.

https://review.rdoproject.org/zuul/buildset/aa62ec29eb1a43b3a7f0159fd8a2a476

:heavy_check_mark: openstack-k8s-operators-content-provider SUCCESS in 3h 27m 03s :x: podified-multinode-edpm-deployment-crc FAILURE in 2h 07m 54s :heavy_check_mark: cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 58m 12s :heavy_check_mark: edpm-ansible-molecule-edpm_bootstrap SUCCESS in 6m 05s :heavy_check_mark: edpm-ansible-molecule-edpm_podman SUCCESS in 5m 10s :heavy_check_mark: edpm-ansible-molecule-edpm_module_load SUCCESS in 4m 35s :heavy_check_mark: edpm-ansible-molecule-edpm_kernel SUCCESS in 9m 25s :heavy_check_mark: edpm-ansible-molecule-edpm_libvirt SUCCESS in 8m 20s :heavy_check_mark: edpm-ansible-molecule-edpm_nova SUCCESS in 8m 35s :heavy_check_mark: edpm-ansible-molecule-edpm_frr SUCCESS in 6m 19s :heavy_check_mark: edpm-ansible-molecule-edpm_iscsid SUCCESS in 4m 34s :heavy_check_mark: edpm-ansible-molecule-edpm_ovn_bgp_agent SUCCESS in 6m 48s :heavy_check_mark: edpm-ansible-molecule-edpm_ovs SUCCESS in 5m 08s :heavy_check_mark: edpm-ansible-molecule-edpm_tripleo_cleanup SUCCESS in 3m 53s :x: cifmw-data-plane-adoption-osp-17-to-extracted-crc FAILURE in 2h 22m 43s

jpodivin commented 3 months ago

+2 from me. Unless there are further objections I think this is the best option we have.

softwarefactory-project-zuul[bot] commented 3 months ago

Build failed (check pipeline). Post recheck (without leading slash) to rerun all jobs. Make sure the failure cause has been resolved before you rerun jobs.

https://review.rdoproject.org/zuul/buildset/ed39e15be1a441c3b0de3753308ceb12

:heavy_check_mark: openstack-k8s-operators-content-provider SUCCESS in 3h 10m 25s :x: podified-multinode-edpm-deployment-crc FAILURE in 2h 09m 42s :heavy_check_mark: cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 55m 55s :heavy_check_mark: edpm-ansible-molecule-edpm_bootstrap SUCCESS in 6m 17s :heavy_check_mark: edpm-ansible-molecule-edpm_podman SUCCESS in 5m 22s :heavy_check_mark: edpm-ansible-molecule-edpm_module_load SUCCESS in 4m 38s :heavy_check_mark: edpm-ansible-molecule-edpm_kernel SUCCESS in 9m 29s :heavy_check_mark: edpm-ansible-molecule-edpm_libvirt SUCCESS in 8m 37s :heavy_check_mark: edpm-ansible-molecule-edpm_nova SUCCESS in 8m 44s :heavy_check_mark: edpm-ansible-molecule-edpm_frr SUCCESS in 6m 37s :heavy_check_mark: edpm-ansible-molecule-edpm_iscsid SUCCESS in 4m 37s :heavy_check_mark: edpm-ansible-molecule-edpm_ovn_bgp_agent SUCCESS in 7m 00s :heavy_check_mark: edpm-ansible-molecule-edpm_ovs SUCCESS in 5m 07s :heavy_check_mark: edpm-ansible-molecule-edpm_tripleo_cleanup SUCCESS in 4m 06s :x: cifmw-data-plane-adoption-osp-17-to-extracted-crc FAILURE in 2h 36m 22s

marios commented 3 months ago

job is hitting https://issues.redhat.com/browse/OSPCIX-318

should be fixed today

https://logserver.rdoproject.org/62/662/52765469d665541b6f449f8f75f8fb06e93c321c/github-check/cifmw-data-plane-adoption-osp-17-to-extracted-crc/67523c6/controller/data-plane-adoption-tests-repo/data-plane-adoption/tests/logs/test_with_ceph_out_2024-06-05T14:38:11EDT.log

TASK [heat_adoption : deploy podified heat] ************************************
...
"The OpenStackControlPlane \"openstack\" is invalid: spec.databaseInstance: Forbidden: Changing the DatabaseInstance is not supported for existing deployments"], "stdout": "", "stdout_lines": []}
marios commented 3 months ago

check-rdo

adoption jobs were fixed/running green yesterday

softwarefactory-project-zuul[bot] commented 3 months ago

Build failed (check pipeline). Post recheck (without leading slash) to rerun all jobs. Make sure the failure cause has been resolved before you rerun jobs.

https://review.rdoproject.org/zuul/buildset/fd374de8cce44ef2b140d5a13c8f99f5

:heavy_check_mark: openstack-k8s-operators-content-provider SUCCESS in 4h 12m 24s :x: podified-multinode-edpm-deployment-crc FAILURE in 2h 13m 17s :x: cifmw-crc-podified-edpm-baremetal FAILURE in 2h 25m 29s :heavy_check_mark: edpm-ansible-molecule-edpm_bootstrap SUCCESS in 6m 13s :heavy_check_mark: edpm-ansible-molecule-edpm_podman SUCCESS in 5m 10s :heavy_check_mark: edpm-ansible-molecule-edpm_module_load SUCCESS in 4m 34s :heavy_check_mark: edpm-ansible-molecule-edpm_kernel SUCCESS in 9m 17s :heavy_check_mark: edpm-ansible-molecule-edpm_libvirt SUCCESS in 8m 13s :heavy_check_mark: edpm-ansible-molecule-edpm_nova SUCCESS in 8m 57s :heavy_check_mark: edpm-ansible-molecule-edpm_frr SUCCESS in 6m 18s :heavy_check_mark: edpm-ansible-molecule-edpm_iscsid SUCCESS in 4m 35s :heavy_check_mark: edpm-ansible-molecule-edpm_ovn_bgp_agent SUCCESS in 6m 43s :heavy_check_mark: edpm-ansible-molecule-edpm_ovs SUCCESS in 4m 59s :heavy_check_mark: edpm-ansible-molecule-edpm_tripleo_cleanup SUCCESS in 3m 58s :heavy_check_mark: cifmw-data-plane-adoption-osp-17-to-extracted-crc SUCCESS in 3h 07m 39s

marios commented 3 months ago

hold rechecks for now please the podified edpm jobs are hitting https://issues.redhat.com/browse/OSPCIX-328

https://logserver.rdoproject.org/62/662/52765469d665541b6f449f8f75f8fb06e93c321c/github-check/cifmw-crc-podified-edpm-baremetal/b247e00/job-output.txt

2024-06-07 06:24:57.582767 | controller | TASK [edpm_deploy_baremetal : Wait for OpenStackDataPlaneNodeSet to be deployed _raw_params=oc wait OpenStackDataPlaneNodeSet {{ cr_name }} --namespace={{ cifmw_install_yamls_defaults['NAMESPACE'] }} --for=condition=ready --timeout={{ cifmw_edpm_deploy_baremetal_wait_dataplane_timeout_mins }}m] ***
2024-06-07 06:24:57.582774 | controller | Friday 07 June 2024  06:24:57 -0400 (0:00:00.037)       0:44:10.831 ***********
2024-06-07 07:04:57.949268 | controller | fatal: [localhost]: FAILED! => changed=true
marios commented 3 months ago

check-rdo

edpm jobs should be fixed now (https://issues.redhat.com/browse/OSPCIX-328)

softwarefactory-project-zuul[bot] commented 3 months ago

Build failed (check pipeline). Post recheck (without leading slash) to rerun all jobs. Make sure the failure cause has been resolved before you rerun jobs.

https://review.rdoproject.org/zuul/buildset/c5b157e9c4b3421aba66dc6095b57208

:heavy_check_mark: openstack-k8s-operators-content-provider SUCCESS in 3h 58m 44s :heavy_check_mark: podified-multinode-edpm-deployment-crc SUCCESS in 1h 43m 06s :heavy_check_mark: cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 51m 54s :heavy_check_mark: edpm-ansible-molecule-edpm_bootstrap SUCCESS in 6m 19s :heavy_check_mark: edpm-ansible-molecule-edpm_podman SUCCESS in 5m 28s :heavy_check_mark: edpm-ansible-molecule-edpm_module_load SUCCESS in 4m 34s :heavy_check_mark: edpm-ansible-molecule-edpm_kernel SUCCESS in 9m 35s :heavy_check_mark: edpm-ansible-molecule-edpm_libvirt SUCCESS in 8m 53s :heavy_check_mark: edpm-ansible-molecule-edpm_nova SUCCESS in 9m 18s :heavy_check_mark: edpm-ansible-molecule-edpm_frr SUCCESS in 6m 16s :heavy_check_mark: edpm-ansible-molecule-edpm_iscsid SUCCESS in 4m 50s :heavy_check_mark: edpm-ansible-molecule-edpm_ovn_bgp_agent SUCCESS in 7m 17s :heavy_check_mark: edpm-ansible-molecule-edpm_ovs SUCCESS in 5m 21s :heavy_check_mark: edpm-ansible-molecule-edpm_tripleo_cleanup SUCCESS in 4m 01s :x: cifmw-data-plane-adoption-osp-17-to-extracted-crc FAILURE in 2h 41m 14s

marios commented 3 months ago

and now blocked on new cix :/ https://issues.redhat.com/browse/OSPCIX-332

[edited to correct cix previously pointed to 328]

marios commented 3 months ago

please hold recheck until we have https://github.com/openstack-k8s-operators/data-plane-adoption/pull/486 (and https://review.rdoproject.org/r/c/rdo-jobs/+/53425 merged) for OSPCIX-332

marios commented 3 months ago

check-rdo

softwarefactory-project-zuul[bot] commented 3 months ago

Build failed (check pipeline). Post recheck (without leading slash) to rerun all jobs. Make sure the failure cause has been resolved before you rerun jobs.

https://review.rdoproject.org/zuul/buildset/84a8a96cecb2453085865b9f8ce083ac

:heavy_check_mark: openstack-k8s-operators-content-provider SUCCESS in 3h 30m 44s :heavy_check_mark: podified-multinode-edpm-deployment-crc SUCCESS in 1h 47m 04s :heavy_check_mark: cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 51m 10s :heavy_check_mark: edpm-ansible-molecule-edpm_bootstrap SUCCESS in 6m 37s :heavy_check_mark: edpm-ansible-molecule-edpm_podman SUCCESS in 5m 13s :heavy_check_mark: edpm-ansible-molecule-edpm_module_load SUCCESS in 4m 36s :heavy_check_mark: edpm-ansible-molecule-edpm_kernel SUCCESS in 10m 11s :heavy_check_mark: edpm-ansible-molecule-edpm_libvirt SUCCESS in 8m 22s :heavy_check_mark: edpm-ansible-molecule-edpm_nova SUCCESS in 8m 43s :heavy_check_mark: edpm-ansible-molecule-edpm_frr SUCCESS in 6m 36s :heavy_check_mark: edpm-ansible-molecule-edpm_iscsid SUCCESS in 4m 51s :heavy_check_mark: edpm-ansible-molecule-edpm_ovn_bgp_agent SUCCESS in 7m 12s :heavy_check_mark: edpm-ansible-molecule-edpm_ovs SUCCESS in 5m 31s :heavy_check_mark: edpm-ansible-molecule-edpm_tripleo_cleanup SUCCESS in 4m 15s :x: cifmw-data-plane-adoption-osp-17-to-extracted-crc FAILURE in 2h 47m 09s

marios commented 3 months ago

check-rdo

should be green now after https://review.rdoproject.org/r/c/rdo-jobs/+/53473

marios commented 3 months ago

green run ready for merge please https://review.rdoproject.org/zuul/buildset/8aac08289be24cf1a2797dbcde434324

rebtoor commented 3 months ago

/approve

openshift-ci[bot] commented 3 months ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bogdando, cescgina, marios, rebtoor

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/openstack-k8s-operators/edpm-ansible/blob/main/OWNERS)~~ [rebtoor] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment