search

canonical / opensearch-operator

OpenSearch operator
Apache License 2.0
10 stars 6 forks source link

Expected sysctl parameters should be enforced by the charm #359

Open sajoupa opened 1 month ago

sajoupa commented 1 month ago

Steps to reproduce

  1. juju deploy ch:opensearch on a substrate that doesn't set vm.max_map_count, vm.swappiness and net.ipv4.tcp_retries2
  2. Let the unit(s) settle, and observe their eventual state

Expected behavior

Opensearch is up and running. If needed, the charm has configured sysctl and applied the required parameters.

Actual behavior

opensearch/3              blocked   idle   2        <private IP>              vm.max_map_count should be at least 262144 - vm.swappiness should be 0 - net.ipv4.tcp_retries2 should be 5
opensearch/4*             blocked   idle   3        <private IP>             vm.max_map_count should be at least 262144 - vm.swappiness should be 0 - net.ipv4.tcp_retries2 should be 5
opensearch/5              blocked   idle   4        <private IP>             vm.max_map_count should be at least 262144 - vm.swappiness should be 0 - net.ipv4.tcp_retries2 should be 5

Versions

Operating system: Ubuntu 22.04.4 LTS

Juju CLI: 3.4.4-ubuntu-amd64

Juju agent: 3.4.4

Charm revision: 113 (2/edge)

LXD: N/A (openstack)

Log output

Juju debug log: Not provided due to the noise from the other units, the snippet below is more relevant.

2024-07-09 15:09:28 INFO juju unit_agent.go:289 Starting unit workers for "opensearch/3"
2024-07-09 15:09:28 INFO juju.worker.apicaller connect.go:163 [658f41] "unit-opensearch-3" successfully connected to "<controller IP>:17070"
2024-07-09 15:09:28 INFO juju.worker.apicaller connect.go:260 [658f41] password changed for "unit-opensearch-3"
2024-07-09 15:09:28 INFO juju.worker.apicaller connect.go:163 [658f41] "unit-opensearch-3" successfully connected to "<controller IP>:17070"
2024-07-09 15:09:28 INFO juju.worker.upgrader upgrader.go:141 no waiter, upgrader is done
2024-07-09 15:09:28 INFO juju.worker.migrationminion worker.go:147 migration phase is now: NONE
2024-07-09 15:09:28 INFO juju.worker.logger logger.go:120 logger worker started
2024-07-09 15:09:28 ERROR juju.worker.meterstatus runner.go:93 error running "meter-status-changed": charm missing from disk
2024-07-09 15:09:28 INFO juju.worker.uniter uniter.go:363 unit "opensearch/3" started
2024-07-09 15:09:28 INFO juju.worker.uniter uniter.go:689 resuming charm install
2024-07-09 15:09:28 INFO juju.worker.uniter.charm bundles.go:82 downloading ch:amd64/jammy/opensearch-113 from API server
2024-07-09 15:09:42 INFO juju.worker.uniter uniter.go:389 hooks are retried true
2024-07-09 15:09:43 DEBUG unit.opensearch/3.juju-log server.go:325 ops 2.14.1 up and running.
2024-07-09 15:09:43 DEBUG unit.opensearch/3.juju-log server.go:325 Using local storage: not a Kubernetes podspec charm
2024-07-09 15:09:43 DEBUG unit.opensearch/3.juju-log server.go:325 Initializing SQLite local storage: /var/lib/juju/agents/unit-opensearch-3/charm/.unit-state.db.
2024-07-09 15:09:44 DEBUG unit.opensearch/3.juju-log server.go:325 Emitting Juju event opensearch_data_storage_attached.
2024-07-09 15:09:44 INFO juju.worker.uniter.operation runhook.go:186 ran "opensearch-data-storage-attached" hook (via hook dispatching script: dispatch)
2024-07-09 15:09:44 INFO juju.worker.uniter.storage resolver.go:126 initial storage attachments ready
2024-07-09 15:09:45 DEBUG unit.opensearch/3.juju-log server.go:325 ops 2.14.1 up and running.
2024-07-09 15:09:45 INFO unit.opensearch/3.juju-log server.go:325 Running legacy hooks/install.
2024-07-09 15:09:45 DEBUG unit.opensearch/3.juju-log server.go:325 ops 2.14.1 up and running.
2024-07-09 15:09:45 DEBUG unit.opensearch/3.juju-log server.go:325 Charm called itself via hooks/install.
2024-07-09 15:09:45 DEBUG unit.opensearch/3.juju-log server.go:325 Legacy hooks/install exited with status 0.
2024-07-09 15:09:45 DEBUG unit.opensearch/3.juju-log server.go:325 Emitting Juju event install.
2024-07-09 15:09:45 INFO unit.opensearch/3.juju-log server.go:325 Installing snap opensearch, revision 53, tracking 
2024-07-09 15:11:12 INFO unit.opensearch/3.juju-log server.go:325 The snap installation completed successfully
2024-07-09 15:11:13 INFO juju.worker.uniter.operation runhook.go:186 ran "install" hook (via hook dispatching script: dispatch)
2024-07-09 15:11:15 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: ops 2.14.1 up and running.
2024-07-09 15:11:15 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: Emitting Juju event opensearch_peers_relation_created.
2024-07-09 15:11:15 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: self._app_workload_container_version='53' self._unit_workload_container_versions={}
2024-07-09 15:11:16 INFO juju.worker.uniter.operation runhook.go:186 ran "opensearch-peers-relation-created" hook (via hook dispatching script: dispatch)
2024-07-09 15:11:16 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: ops 2.14.1 up and running.
2024-07-09 15:11:16 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Emitting Juju event upgrade_version_a_relation_created.
2024-07-09 15:11:16 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Saved _SNAP_REVISION='53' and self._current_versions["workload"]='2.14.0' in unit databag after first install
2024-07-09 15:11:17 INFO juju.worker.uniter.operation runhook.go:186 ran "upgrade-version-a-relation-created" hook (via hook dispatching script: dispatch)
2024-07-09 15:13:40 DEBUG unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: ops 2.14.1 up and running.
2024-07-09 15:13:40 DEBUG unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: Emitting Juju event node_lock_fallback_relation_created.
2024-07-09 15:13:40 INFO juju.worker.uniter.operation runhook.go:186 ran "node-lock-fallback-relation-created" hook (via hook dispatching script: dispatch)
2024-07-09 15:13:43 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-client:18: ops 2.14.1 up and running.
2024-07-09 15:13:43 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-client:18: Emitting Juju event opensearch_client_relation_created.
2024-07-09 15:13:43 INFO juju.worker.uniter.operation runhook.go:186 ran "opensearch-client-relation-created" hook (via hook dispatching script: dispatch)
2024-07-09 15:13:47 DEBUG unit.opensearch/3.juju-log server.go:325 ops 2.14.1 up and running.
2024-07-09 15:13:47 DEBUG unit.opensearch/3.juju-log server.go:325 Emitting Juju event leader_settings_changed.
2024-07-09 15:13:47 INFO juju.worker.uniter.operation runhook.go:186 ran "leader-settings-changed" hook (via hook dispatching script: dispatch)
2024-07-09 15:13:52 DEBUG unit.opensearch/3.juju-log server.go:325 ops 2.14.1 up and running.
2024-07-09 15:13:52 DEBUG unit.opensearch/3.juju-log server.go:325 Emitting Juju event config_changed.
2024-07-09 15:13:53 INFO juju.worker.uniter.operation runhook.go:186 ran "config-changed" hook (via hook dispatching script: dispatch)
2024-07-09 15:13:53 INFO juju.worker.uniter resolver.go:185 found queued "start" hook
2024-07-09 15:14:08 DEBUG unit.opensearch/3.juju-log server.go:325 ops 2.14.1 up and running.
2024-07-09 15:14:08 INFO unit.opensearch/3.juju-log server.go:325 Running legacy hooks/start.
2024-07-09 15:14:08 DEBUG unit.opensearch/3.juju-log server.go:325 ops 2.14.1 up and running.
2024-07-09 15:14:08 DEBUG unit.opensearch/3.juju-log server.go:325 Charm called itself via hooks/start.
2024-07-09 15:14:08 DEBUG unit.opensearch/3.juju-log server.go:325 Legacy hooks/start exited with status 0.
2024-07-09 15:14:08 DEBUG unit.opensearch/3.juju-log server.go:325 Emitting Juju event start.
2024-07-09 15:14:08 ERROR unit.opensearch/3.juju-log server.go:325 [Errno 111] Connection refused
2024-07-09 15:14:09 DEBUG unit.opensearch/3.juju-log server.go:325 Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:09 INFO juju.worker.uniter.operation runhook.go:186 ran "start" hook (via hook dispatching script: dispatch)
2024-07-09 15:14:23 DEBUG unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: ops 2.14.1 up and running.
2024-07-09 15:14:23 DEBUG unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:23 ERROR unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: [Errno 111] Connection refused
2024-07-09 15:14:23 DEBUG unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:23 DEBUG unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: Emitting Juju event node_lock_fallback_relation_joined.
2024-07-09 15:14:23 INFO juju.worker.uniter.operation runhook.go:186 ran "node-lock-fallback-relation-joined" hook (via hook dispatching script: dispatch)
2024-07-09 15:14:27 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: ops 2.14.1 up and running.
2024-07-09 15:14:27 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:27 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:27 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:27 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: Emitting Juju event opensearch_peers_relation_changed.
2024-07-09 15:14:27 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:28 INFO juju.worker.uniter.operation runhook.go:186 ran "opensearch-peers-relation-changed" hook (via hook dispatching script: dispatch)
2024-07-09 15:14:28 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: ops 2.14.1 up and running.
2024-07-09 15:14:28 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:28 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:28 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:28 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:28 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: Emitting Juju event opensearch_peers_relation_joined.
2024-07-09 15:14:28 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: self._app_workload_container_version='53' self._unit_workload_container_versions={'opensearch/3': '53'}
2024-07-09 15:14:29 INFO juju.worker.uniter.operation runhook.go:186 ran "opensearch-peers-relation-joined" hook (via hook dispatching script: dispatch)
2024-07-09 15:14:29 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: ops 2.14.1 up and running.
2024-07-09 15:14:29 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:29 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:29 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:30 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:30 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: Emitting Juju event opensearch_peers_relation_changed.
2024-07-09 15:14:30 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:30 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:30 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:30 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:30 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:30 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:30 INFO juju.worker.uniter.operation runhook.go:186 ran "opensearch-peers-relation-changed" hook (via hook dispatching script: dispatch)
2024-07-09 15:14:31 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: ops 2.14.1 up and running.
2024-07-09 15:14:31 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:31 ERROR unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: [Errno 111] Connection refused
2024-07-09 15:14:31 ERROR unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: [Errno 111] Connection refused
2024-07-09 15:14:31 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:31 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Emitting Juju event upgrade_version_a_relation_changed.
2024-07-09 15:14:31 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Versions before upgrade compatible with versions after upgrade previous_version_strs={'charm': '1', 'workload': '2.14.0'} self._current_versions={'charm': '1+3007c1a', 'workload': '2.14.0'}
2024-07-09 15:14:31 INFO juju.worker.uniter.operation runhook.go:186 ran "upgrade-version-a-relation-changed" hook (via hook dispatching script: dispatch)
2024-07-09 15:14:32 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: ops 2.14.1 up and running.
2024-07-09 15:14:32 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:32 ERROR unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: [Errno 111] Connection refused
2024-07-09 15:14:32 ERROR unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: [Errno 111] Connection refused
2024-07-09 15:14:32 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:32 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Emitting Juju event upgrade_version_a_relation_joined.
2024-07-09 15:14:32 INFO juju.worker.uniter.operation runhook.go:186 ran "upgrade-version-a-relation-joined" hook (via hook dispatching script: dispatch)
2024-07-09 15:14:44 DEBUG unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: ops 2.14.1 up and running.
2024-07-09 15:14:44 DEBUG unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:44 ERROR unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: [Errno 111] Connection refused
2024-07-09 15:14:44 ERROR unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: [Errno 111] Connection refused
2024-07-09 15:14:44 DEBUG unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:44 DEBUG unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: Emitting Juju event node_lock_fallback_relation_changed.
2024-07-09 15:14:44 INFO juju.worker.uniter.operation runhook.go:186 ran "node-lock-fallback-relation-changed" hook (via hook dispatching script: dispatch)
2024-07-09 15:14:45 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: ops 2.14.1 up and running.
2024-07-09 15:14:45 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:45 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:45 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:45 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:45 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:45 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: Emitting Juju event opensearch_peers_relation_joined.
2024-07-09 15:14:45 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: self._app_workload_container_version='53' self._unit_workload_container_versions={'opensearch/4': '53', 'opensearch/3': '53'}
2024-07-09 15:14:45 INFO juju.worker.uniter.operation runhook.go:186 ran "opensearch-peers-relation-joined" hook (via hook dispatching script: dispatch)
2024-07-09 15:14:46 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: ops 2.14.1 up and running.
2024-07-09 15:14:46 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:46 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:46 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:46 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:46 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:46 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: Emitting Juju event opensearch_peers_relation_changed.
2024-07-09 15:14:46 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:46 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:46 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:46 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:46 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:46 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:46 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:46 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:46 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-peers:9: [Errno 111] Connection refused
2024-07-09 15:14:46 INFO juju.worker.uniter.operation runhook.go:186 ran "opensearch-peers-relation-changed" hook (via hook dispatching script: dispatch)
2024-07-09 15:14:47 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: ops 2.14.1 up and running.
2024-07-09 15:14:47 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:47 ERROR unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: [Errno 111] Connection refused
2024-07-09 15:14:47 ERROR unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: [Errno 111] Connection refused
2024-07-09 15:14:47 ERROR unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: [Errno 111] Connection refused
2024-07-09 15:14:47 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:47 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Emitting Juju event upgrade_version_a_relation_changed.
2024-07-09 15:14:47 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Versions before upgrade compatible with versions after upgrade previous_version_strs={'charm': '1', 'workload': '2.14.0'} self._current_versions={'charm': '1+3007c1a', 'workload': '2.14.0'}
2024-07-09 15:14:48 INFO juju.worker.uniter.operation runhook.go:186 ran "upgrade-version-a-relation-changed" hook (via hook dispatching script: dispatch)
2024-07-09 15:14:48 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: ops 2.14.1 up and running.
2024-07-09 15:14:48 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:48 ERROR unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: [Errno 111] Connection refused
2024-07-09 15:14:48 ERROR unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: [Errno 111] Connection refused
2024-07-09 15:14:48 ERROR unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: [Errno 111] Connection refused
2024-07-09 15:14:48 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:48 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Emitting Juju event upgrade_version_a_relation_joined.
2024-07-09 15:14:49 INFO juju.worker.uniter.operation runhook.go:186 ran "upgrade-version-a-relation-joined" hook (via hook dispatching script: dispatch)
2024-07-09 15:14:49 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: ops 2.14.1 up and running.
2024-07-09 15:14:49 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:49 ERROR unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: [Errno 111] Connection refused
2024-07-09 15:14:49 ERROR unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: [Errno 111] Connection refused
2024-07-09 15:14:49 ERROR unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: [Errno 111] Connection refused
2024-07-09 15:14:49 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:49 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Emitting Juju event upgrade_version_a_relation_changed.
2024-07-09 15:14:49 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Versions before upgrade compatible with versions after upgrade previous_version_strs={'charm': '1', 'workload': '2.14.0'} self._current_versions={'charm': '1+3007c1a', 'workload': '2.14.0'}
2024-07-09 15:14:50 INFO juju.worker.uniter.operation runhook.go:186 ran "upgrade-version-a-relation-changed" hook (via hook dispatching script: dispatch)
2024-07-09 15:14:50 DEBUG unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: ops 2.14.1 up and running.
2024-07-09 15:14:50 DEBUG unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:50 ERROR unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: [Errno 111] Connection refused
2024-07-09 15:14:50 ERROR unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: [Errno 111] Connection refused
2024-07-09 15:14:50 ERROR unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: [Errno 111] Connection refused
2024-07-09 15:14:50 DEBUG unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:50 DEBUG unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: Emitting Juju event node_lock_fallback_relation_joined.
2024-07-09 15:14:51 INFO juju.worker.uniter.operation runhook.go:186 ran "node-lock-fallback-relation-joined" hook (via hook dispatching script: dispatch)
2024-07-09 15:14:51 DEBUG unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: ops 2.14.1 up and running.
2024-07-09 15:14:52 DEBUG unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:52 ERROR unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: [Errno 111] Connection refused
2024-07-09 15:14:52 ERROR unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: [Errno 111] Connection refused
2024-07-09 15:14:52 ERROR unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: [Errno 111] Connection refused
2024-07-09 15:14:52 DEBUG unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:14:52 DEBUG unit.opensearch/3.juju-log server.go:325 node-lock-fallback:11: Emitting Juju event node_lock_fallback_relation_changed.
2024-07-09 15:14:52 INFO juju.worker.uniter.operation runhook.go:186 ran "node-lock-fallback-relation-changed" hook (via hook dispatching script: dispatch)
2024-07-09 15:15:01 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: ops 2.14.1 up and running.
2024-07-09 15:15:01 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:15:01 ERROR unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: [Errno 111] Connection refused
2024-07-09 15:15:01 ERROR unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: [Errno 111] Connection refused
2024-07-09 15:15:01 ERROR unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: [Errno 111] Connection refused
2024-07-09 15:15:02 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:15:02 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Emitting Juju event upgrade_version_a_relation_changed.
2024-07-09 15:15:02 DEBUG unit.opensearch/3.juju-log server.go:325 upgrade-version-a:10: Versions before upgrade compatible with versions after upgrade previous_version_strs={'charm': '1', 'workload': '2.14.0'} self._current_versions={'charm': '1+3007c1a', 'workload': '2.14.0'}
2024-07-09 15:15:02 INFO juju.worker.uniter.operation runhook.go:186 ran "upgrade-version-a-relation-changed" hook (via hook dispatching script: dispatch)
2024-07-09 15:15:20 DEBUG unit.opensearch/3.juju-log server.go:325 ops 2.14.1 up and running.
2024-07-09 15:15:20 DEBUG unit.opensearch/3.juju-log server.go:325 Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:15:21 ERROR unit.opensearch/3.juju-log server.go:325 [Errno 111] Connection refused
2024-07-09 15:15:21 ERROR unit.opensearch/3.juju-log server.go:325 [Errno 111] Connection refused
2024-07-09 15:15:21 ERROR unit.opensearch/3.juju-log server.go:325 [Errno 111] Connection refused
2024-07-09 15:15:21 DEBUG unit.opensearch/3.juju-log server.go:325 Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:15:21 DEBUG unit.opensearch/3.juju-log server.go:325 Emitting Juju event update_status.
2024-07-09 15:15:21 INFO juju.worker.uniter.operation runhook.go:186 ran "update-status" hook (via hook dispatching script: dispatch)
2024-07-09 15:20:12 DEBUG unit.opensearch/3.juju-log server.go:325 ops 2.14.1 up and running.
2024-07-09 15:20:12 DEBUG unit.opensearch/3.juju-log server.go:325 Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:20:12 ERROR unit.opensearch/3.juju-log server.go:325 [Errno 111] Connection refused
2024-07-09 15:20:12 ERROR unit.opensearch/3.juju-log server.go:325 [Errno 111] Connection refused
2024-07-09 15:20:12 ERROR unit.opensearch/3.juju-log server.go:325 [Errno 111] Connection refused
2024-07-09 15:20:12 DEBUG unit.opensearch/3.juju-log server.go:325 Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:20:12 DEBUG unit.opensearch/3.juju-log server.go:325 Emitting Juju event update_status.
2024-07-09 15:20:13 INFO juju.worker.uniter.operation runhook.go:186 ran "update-status" hook (via hook dispatching script: dispatch)
2024-07-09 15:22:54 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-client:18: ops 2.14.1 up and running.
2024-07-09 15:22:55 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-client:18: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:22:55 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-client:18: [Errno 111] Connection refused
2024-07-09 15:22:55 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-client:18: [Errno 111] Connection refused
2024-07-09 15:22:55 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-client:18: [Errno 111] Connection refused
2024-07-09 15:22:55 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-client:18: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:22:55 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-client:18: Emitting Juju event opensearch_client_relation_joined.
2024-07-09 15:22:55 INFO juju.worker.uniter.operation runhook.go:186 ran "opensearch-client-relation-joined" hook (via hook dispatching script: dispatch)
2024-07-09 15:22:56 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-client:18: ops 2.14.1 up and running.
2024-07-09 15:22:56 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-client:18: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:22:56 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-client:18: [Errno 111] Connection refused
2024-07-09 15:22:56 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-client:18: [Errno 111] Connection refused
2024-07-09 15:22:56 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-client:18: [Errno 111] Connection refused
2024-07-09 15:22:56 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-client:18: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:22:56 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-client:18: Emitting Juju event opensearch_client_relation_changed.
2024-07-09 15:22:56 INFO juju.worker.uniter.operation runhook.go:186 ran "opensearch-client-relation-changed" hook (via hook dispatching script: dispatch)
2024-07-09 15:22:58 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-client:18: ops 2.14.1 up and running.
2024-07-09 15:22:58 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-client:18: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:22:58 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-client:18: [Errno 111] Connection refused
2024-07-09 15:22:58 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-client:18: [Errno 111] Connection refused
2024-07-09 15:22:58 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-client:18: [Errno 111] Connection refused
2024-07-09 15:22:58 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-client:18: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:22:58 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-client:18: Emitting Juju event opensearch_client_relation_changed.
2024-07-09 15:22:59 INFO juju.worker.uniter.operation runhook.go:186 ran "opensearch-client-relation-changed" hook (via hook dispatching script: dispatch)
2024-07-09 15:23:16 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-client:18: ops 2.14.1 up and running.
2024-07-09 15:23:16 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-client:18: Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:23:16 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-client:18: [Errno 111] Connection refused
2024-07-09 15:23:16 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-client:18: [Errno 111] Connection refused
2024-07-09 15:23:16 ERROR unit.opensearch/3.juju-log server.go:325 opensearch-client:18: [Errno 111] Connection refused
2024-07-09 15:23:16 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-client:18: Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:23:16 DEBUG unit.opensearch/3.juju-log server.go:325 opensearch-client:18: Emitting Juju event opensearch_client_relation_changed.
2024-07-09 15:23:16 INFO juju.worker.uniter.operation runhook.go:186 ran "opensearch-client-relation-changed" hook (via hook dispatching script: dispatch)
2024-07-09 15:29:07 DEBUG unit.opensearch/3.juju-log server.go:325 ops 2.14.1 up and running.
2024-07-09 15:29:07 DEBUG unit.opensearch/3.juju-log server.go:325 Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:29:08 ERROR unit.opensearch/3.juju-log server.go:325 [Errno 111] Connection refused
2024-07-09 15:29:08 ERROR unit.opensearch/3.juju-log server.go:325 [Errno 111] Connection refused
2024-07-09 15:29:08 ERROR unit.opensearch/3.juju-log server.go:325 [Errno 111] Connection refused
2024-07-09 15:29:08 DEBUG unit.opensearch/3.juju-log server.go:325 Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:29:08 DEBUG unit.opensearch/3.juju-log server.go:325 Emitting Juju event update_status.
2024-07-09 15:29:08 INFO juju.worker.uniter.operation runhook.go:186 ran "update-status" hook (via hook dispatching script: dispatch)
2024-07-09 15:33:53 DEBUG unit.opensearch/3.juju-log server.go:325 ops 2.14.1 up and running.
2024-07-09 15:33:53 DEBUG unit.opensearch/3.juju-log server.go:325 Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:33:53 ERROR unit.opensearch/3.juju-log server.go:325 [Errno 111] Connection refused
2024-07-09 15:33:53 ERROR unit.opensearch/3.juju-log server.go:325 [Errno 111] Connection refused
2024-07-09 15:33:54 ERROR unit.opensearch/3.juju-log server.go:325 [Errno 111] Connection refused
2024-07-09 15:33:54 DEBUG unit.opensearch/3.juju-log server.go:325 Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:33:54 DEBUG unit.opensearch/3.juju-log server.go:325 Emitting Juju event update_status.
2024-07-09 15:33:54 INFO juju.worker.uniter.operation runhook.go:186 ran "update-status" hook (via hook dispatching script: dispatch)
2024-07-09 15:39:23 DEBUG unit.opensearch/3.juju-log server.go:325 ops 2.14.1 up and running.
2024-07-09 15:39:23 DEBUG unit.opensearch/3.juju-log server.go:325 Re-emitting deferred event <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:39:23 ERROR unit.opensearch/3.juju-log server.go:325 [Errno 111] Connection refused
2024-07-09 15:39:23 ERROR unit.opensearch/3.juju-log server.go:325 [Errno 111] Connection refused
2024-07-09 15:39:23 ERROR unit.opensearch/3.juju-log server.go:325 [Errno 111] Connection refused
2024-07-09 15:39:24 DEBUG unit.opensearch/3.juju-log server.go:325 Deferring <StartEvent via OpenSearchOperatorCharm/on/start[33]>.
2024-07-09 15:39:24 DEBUG unit.opensearch/3.juju-log server.go:325 Emitting Juju event update_status.
2024-07-09 15:39:24 INFO juju.worker.uniter.operation runhook.go:186 ran "update-status" hook (via hook dispatching script: dispatch)

Additional context

github-actions[bot] commented 1 month ago

https://warthogs.atlassian.net/browse/DPE-4859

sajoupa commented 1 month ago

I just saw that the documentation recommends to set these parameters at the model-config level with cloud-init. This isn't really convenient, as the parameters will apply to all other units of the environment (frontends, ETL units, anything, ...)

phvalguima commented 1 month ago

@sajoupa we can use the sysconfig then, and relate it to OpenSearch.