canonical / charm-openstack-service-checks

Collection of Nagios checks and other utilities that can be used to verify the operation of an OpenStack cluster
0 stars 2 forks source link

[focal][yoga][quincy][juju 3.1.5] installation failed on config-get --all --format=json #154

Closed sudeephb closed 6 months ago

sudeephb commented 6 months ago

During deployment juju-wait is aborted as o-s-c reports openstack-service-checks does not have a leader

Logs show it fails with

subprocess.CalledProcessError: Command '['config-get', '--all', '--format=json']' returned non-zero exit status 1

logs and artifacts

https://oil-jenkins.canonical.com/artifacts/91916db1-1ae1-429a-a93e-fb588beb4a7e/index.html


Imported from Launchpad using lp2gh.

sudeephb commented 6 months ago

(by ajkavanagh) Please, as a minimum, add:

sudeephb commented 6 months ago

(by marosg) Hi Alex. Issue with o-s-c was

Ubuntu Focal Openstack Yoga Ceph Quincy/stable (17.2.6) Juju 3.1.5

There is juju status visible in fce_build_29255/fce_build_29255_console.out in artifacts if you need more details.

sudeephb commented 6 months ago

(by ajkavanagh) This doesn't appear to be a ceph or openstack-service-checks bug. Perhaps a juju-wait or juju bug. Setting both to invalid.

i.e. the runner dies with with:

INFO:root:openstack-service-checks does not have a leader

Traceback (most recent call last): File "/usr/local/bin/fce", line 33, in sys.exit(load_entry_point('foundationcloudengine', 'console_scripts', 'fce')()) File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/main.py", line 217, in entry_point sys.exit(main(sys.argv[1:])) File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/main.py", line 208, in main opts.func(opts) File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/command.py", line 94, in run self.run_per_layer(layer, args) File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/commands/build.py", line 46, in run_per_layer build_and_validate_if_needed( File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/commands/build.py", line 98, in build_and_validate_if_needed layer.build_outer(only_steps) File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/layers/baselayer.py", line 160, in build_outer self.build(only_steps=only_steps) File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/layers/openstacklayer.py", line 2604, in build super().run_steps(only_steps) File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/layers/steppedbaselayer.py", line 110, in run_steps step.build() File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/layers/jujuworkloadlayer.py", line 1013, in build self.waiter() File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/layers/jujuworkloadlayer.py", line 1746, in wait_until_ready wait( File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/juju_cli.py", line 501, in wait local(command, output_mode="live") File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/remotehelpers.py", line 278, in local return run_cmd(cmd, target_machine, **kwargs) File "/home/ubuntu/cpe/foundation/foundationcloudengine/foundationcloudengine/remotehelpers.py", line 209, in run_cmd raise subprocess.CalledProcessError( subprocess.CalledProcessError: Command '['juju-wait', '-m', 'foundations-maas:openstack', '-t', '14400', '--workload', '--machine-error-timeout', '1800', '--retry_errors', '5', '--verbose', '-x', 'ceph-radosgw', '-x', 'vault', '-x', 'ceph-osd', '-x', 'aw s-k8s-storage', '-x', 'sysconfig', '-x', 'manila-ganesha', '-x', 'filebeat', '-x', 'kubernetes-worker', '-x', 'ceph-mon', '-x', 'ceph-fs', '-x', 'ceilometer', '-x', 'glance', '-x', 'barbican-vault', '-x', 'octavia', '-x', 'ovn-central', '-x', 'cinder-cep h', '-x', 'ubuntu-advantage', '-x', 'kube-ovn', '-x', 'landscape-client', '-x', 'prometheus-openstack-exporter', '-x', 'kubernetes-control-plane', '-x', 'calico', '-x', 'cilium', '-x', 'openstack-service-checks', '-x', 'cos-proxy', '-x', 'ceph-dashboard', '-x', 'prometheus-ceph-exporter', '-x', 'nova-compute', '-x', 'neutron-api-plugin-ovn', '-x', 'octavia-ovn-chassis', '-x', 'ovn-chassis', '-x', 'aws-cloud-provider', '-x', 'gnocchi']' died with <Signals.SIGKILL: 9>. Build step 'Execute shell' marked build as failure Performing Post build task...