In summary, IS raised an issue of too many secret revisions causing a slowdown of a production Juju MongoDB snap, causing an outage.
After debugging, the cause was the Kafka charm using DP libs calling the following every update-status:
After discussion with the Juju team, it appears that every time we secret.set_content(), it creates a new revision, regardless of whether or not the contents have actually changed.
Issue
Too many secret revisions causes issues for the Juju MongoDB snap.
Currently, there are no protections in ops or DP Libs to avoid repeatedly updating a secret field with the same content, which is a fairly reasonable use-case, present in 'Config' and 'Relation Data' setting.
Secrets should work the same way as charm 'Config' and 'Relation Data', where setting the same value ends up in a no-op, doing 'nothing'.
The Juju team said that this won't be possible to implement on their end.
Request
Make DP Libs check secret content during a desired update to a secret field, and if the update is the same as the current content, do nothing.
Desired Behavior
In pseudo, the main gist would be to support the following:
def update_clients(self):
for relation in self.model.relations[CLIENT_RELATION_NAME]:
secret = get_secret(relation.name) # let's assume this func exists
secret.set_content({"some-state": self.get_up_to_date_state()})
def _on_update_status(self, event: UpdateStatusEvent):
self.update_clients()
The actual get_secret and set_content is of course abstracted by the lib. But essentially whenever the lib would write a secret, it should check the content and skip it if it's unchanged.
Context
Full context can be found in this Matrix thread
In summary, IS raised an issue of too many secret revisions causing a slowdown of a production Juju MongoDB snap, causing an outage. After debugging, the cause was the Kafka charm using DP libs calling the following every
update-status
:After discussion with the Juju team, it appears that every time we
secret.set_content()
, it creates a new revision, regardless of whether or not the contents have actually changed.Issue
Too many secret revisions causes issues for the Juju MongoDB snap. Currently, there are no protections in
ops
or DP Libs to avoid repeatedly updating a secret field with the same content, which is a fairly reasonable use-case, present in 'Config' and 'Relation Data' setting. Secrets should work the same way as charm 'Config' and 'Relation Data', where setting the same value ends up in a no-op, doing 'nothing'. The Juju team said that this won't be possible to implement on their end.Request
Make DP Libs check secret content during a desired update to a secret field, and if the update is the same as the current content, do nothing.
Desired Behavior
In pseudo, the main gist would be to support the following:
The actual
get_secret
andset_content
is of course abstracted by the lib. But essentially whenever the lib would write a secret, it should check the content and skip it if it's unchanged.