canonical / observability-libs

https://charmhub.io/observability-libs
Apache License 2.0
3 stars 8 forks source link

`private_key` may end up missing from peer data #50

Closed sed-i closed 1 year ago

sed-i commented 1 year ago

Bug Description

Related alertmanager to self-signed-certificate and saw the error. Haven't figured out yet the code path that can lead to private_key not being set.

To Reproduce

  1. Deploy alertmanager (2 units)
  2. Upgrade alertmanager from path.
  3. Relate to self-signed-certificate charm.

Environment

Model                        Controller  Cloud/Region        Version  SLA          Timestamp
test-prometheus-alerts-32k4  uk8s        microk8s/localhost  3.2.0    unsupported  19:10:57-04:00

App   Version  Status   Scale  Charm                     Channel  Rev  Address         Exposed  Message
am    0.25.0   waiting      2  alertmanager-k8s          edge       0  10.152.183.174  no       waiting for units to settle down
ca             active       1  self-signed-certificates  edge       8  10.152.183.38   no       

Unit     Workload  Agent  Address       Ports  Message
am/0     error     idle   10.1.166.78          hook failed: "certificates-relation-joined" for ca:certificates
am/1*    error     idle   10.1.166.80          hook failed: "certificates-relation-joined" for ca:certificates
ca/0*    active    idle   10.1.166.99          

Relevant log output

unit-am-0: 19:06:31.353 DEBUG unit.am/0.juju-log certificates:7: Emitting Juju event certificates_relation_joined.
unit-am-0: 19:06:31.371 ERROR unit.am/0.juju-log certificates:7: Uncaught exception while in charm code:
Traceback (most recent call last):
  File "./src/charm.py", line 540, in <module>
    main(AlertmanagerCharm, use_juju_for_storage=True)
  File "/var/lib/juju/agents/unit-am-0/charm/venv/ops/main.py", line 441, in main
    _emit_charm_event(charm, dispatcher.event_name)
  File "/var/lib/juju/agents/unit-am-0/charm/venv/ops/main.py", line 149, in _emit_charm_event
    event_to_emit.emit(*args, **kwargs)
  File "/var/lib/juju/agents/unit-am-0/charm/venv/ops/framework.py", line 344, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-am-0/charm/venv/ops/framework.py", line 833, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-am-0/charm/venv/ops/framework.py", line 922, in _reemit
    custom_handler(event)
  File "/var/lib/juju/agents/unit-am-0/charm/lib/charms/observability_libs/v0/cert_handler.py", line 182, in _on_certificates_relation_joined
    self._generate_csr()
  File "/var/lib/juju/agents/unit-am-0/charm/lib/charms/observability_libs/v0/cert_handler.py", line 213, in _generate_csr
    assert private_key is not None  # for type checker
AssertionError

Additional context

No response

PietroPasotti commented 1 year ago

just bumped into this as well

PietroPasotti commented 1 year ago

I verified that after syncing the patched branch into catalogue and firing replicas-relation-created, things work as they should