openshift-metal3 / dev-scripts

Scripts to automate development/test setup for openshift integration with https://github.com/metal3-io/
Apache License 2.0
93 stars 185 forks source link

Properly manage libvirt services #1603

Closed cjeanner closed 9 months ago

cjeanner commented 9 months ago

With newer CS9 and RHEL-9, libvirtd has moved to a modular, socket activated layout.

This new layout conflicts with the single, monolithic "libvirtd.service" that was restarted until now, creating issues with other projects that may wrap this metal3-dev-env.

There is a related PR against metal3-env-dev[1].

[1] https://github.com/metal3-io/metal3-dev-env/pull/1313

openshift-ci[bot] commented 9 months ago

Hi @cjeanner. Thanks for your PR.

I'm waiting for a openshift-metal3 member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
dtantsur commented 9 months ago

/ok-to-test

dtantsur commented 9 months ago

/approve /cc @elfosardo @derekhiggins

openshift-ci[bot] commented 9 months ago

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: dtantsur

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files: - ~~[OWNERS](https://github.com/openshift-metal3/dev-scripts/blob/master/OWNERS)~~ [dtantsur] Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment
andfasano commented 9 months ago

/retest-required

cjeanner commented 9 months ago

Hmm I think there's an issue somewhere else in the tests? curl: (7) Failed to connect to ofcir-service.ofcir-system.svc.cluster.local port 443: No route to host not sure it has anything to do with my patch, does it?

The other tests also failed while trying to access cluster.local resources.

andfasano commented 9 months ago

Hmm I think there's an issue somewhere else in the tests? curl: (7) Failed to connect to ofcir-service.ofcir-system.svc.cluster.local port 443: No route to host not sure it has anything to do with my patch, does it?

The other tests also failed while trying to access cluster.local resources.

Yes, that's not directly related to the patch, but it's a CI related problem

cjeanner commented 9 months ago

/test e2e-agent-ha-dualstack

cjeanner commented 9 months ago

@andfasano not really sure about the cause of the failure for the e2e-agent-ha-dualstack :/. Prow isn't that nice in showing logs..

cjeanner commented 9 months ago

Ah, seems to be unrelated to the change:

level=debug msg=OpenShift console route is admitted
level=debug msg=Still waiting for the cluster to initialize: Cluster operators authentication, console, ingress, kube-    apiserver, monitoring are not available
level=debug msg=Still waiting for the cluster to initialize: Cluster operators authentication, console, ingress, monitoring are not available
level=debug msg=Still waiting for the cluster to in    itialize: Cluster operators authentication, console, monitoring are not available
level=debug msg=Still waiting for the cluster to initialize: Cluster operators authentication, console are not available
level=debug msg=Still waiting     for the cluster to initialize: Cluster operator authentication is not available
level=debug msg=Still waiting for the cluster to initialize: Cluster operator etcd is degraded
level=debug msg=Still waiting for the cluster to initializ    e: Cluster operator etcd is degraded
bash: line 190: 17345 Killed                  timeout -s 9 105m make agent

apparently the cluster wasn't ready in time?

andfasano commented 9 months ago

Ah, seems to be unrelated to the change:

level=debug msg=OpenShift console route is admitted
level=debug msg=Still waiting for the cluster to initialize: Cluster operators authentication, console, ingress, kube-    apiserver, monitoring are not available
level=debug msg=Still waiting for the cluster to initialize: Cluster operators authentication, console, ingress, monitoring are not available
level=debug msg=Still waiting for the cluster to in    itialize: Cluster operators authentication, console, monitoring are not available
level=debug msg=Still waiting for the cluster to initialize: Cluster operators authentication, console are not available
level=debug msg=Still waiting     for the cluster to initialize: Cluster operator authentication is not available
level=debug msg=Still waiting for the cluster to initialize: Cluster operator etcd is degraded
level=debug msg=Still waiting for the cluster to initializ    e: Cluster operator etcd is degraded
bash: line 190: 17345 Killed                  timeout -s 9 105m make agent

apparently the cluster wasn't ready in time?

Right, the internal timeout interrupted the job, which took longer due the Cluster operator etcd is degraded problem

andfasano commented 9 months ago

/test e2e-agent-ha-dualstack

cjeanner commented 9 months ago

/hole will move the method definition accordingly

openshift-ci[bot] commented 9 months ago

@cjeanner: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-metal-ipi-bm-bond d3844367737ec62cb649011c316f3fda36e2a933 link false /test e2e-metal-ipi-bm-bond

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
cjeanner commented 9 months ago

/retest-required

andfasano commented 9 months ago

/lgtm