search

canonical / mysql-k8s-operator

A Charmed Operator for running MySQL on Kubernetes
https://charmhub.io/mysql-k8s
Apache License 2.0
8 stars 15 forks source link

single unit mysql stuck at `Primary restart deferred after other units` #416

Closed gboutry closed 2 weeks ago

gboutry commented 1 month ago

Steps to reproduce

  1. Deploy a single unit mysql
  2. juju config mysql profile-limit-memory=3092

Expected behavior

Unit should restart

Actual behavior

image image

Versions

Operating system: Ubuntu 24.04 LTS

Juju CLI: 3.4.2-genericlinux-amd64

Juju agent: 3.4.2

Charm revision: 138 (8.0/candidate)

microk8s: MicroK8s v1.28.7 revision 6532

Log output

2024-05-17T11:19:42.897Z [container-agent] 2024-05-17 11:19:42 INFO juju-log Unit workload member-state is online with member-role primary                                        
2024-05-17T11:19:45.045Z [container-agent] 2024-05-17 11:19:45 INFO juju.worker.uniter.operation runhook.go:186 ran "update-status" hook (via hook dispatching script: dispatch)  
2024-05-17T11:19:55.266Z [container-agent] 2024-05-17 11:19:55 INFO juju-log Configuration change requires restart                                                                
2024-05-17T11:19:55.743Z [container-agent] 2024-05-17 11:19:55 INFO juju-log Primary restart deferred after other units                                                           
2024-05-17T11:19:56.114Z [container-agent] 2024-05-17 11:19:56 INFO juju.worker.uniter.operation runhook.go:186 ran "config-changed" hook (via hook dispatching script: dispatch) 
2024-05-17T11:19:56.989Z [container-agent] 2024-05-17 11:19:56 INFO juju-log restart:86: Primary restart deferred after other units                                               
2024-05-17T11:19:57.349Z [container-agent] 2024-05-17 11:19:57 INFO juju.worker.uniter.operation runhook.go:186 ran "restart-relation-changed" hook (via hook dispatching script: 
2024-05-17T11:20:01.425Z [container-agent] 2024-05-17 11:20:01 INFO juju-log Primary restart deferred after other units                                                           
2024-05-17T11:21:01.447Z [container-agent] 2024-05-17 11:21:01 INFO juju-log Primary restart deferred after other units                                                           
2024-05-17T11:22:01.411Z [container-agent] 2024-05-17 11:22:01 INFO juju-log Primary restart deferred after other units                                                           
2024-05-17T11:23:01.312Z [container-agent] 2024-05-17 11:23:01 INFO juju-log Primary restart deferred after other units                                                           
2024-05-17T11:24:01.578Z [container-agent] 2024-05-17 11:24:01 INFO juju-log Primary restart deferred after other units                                                           
2024-05-17T11:25:01.604Z [container-agent] 2024-05-17 11:25:01 INFO juju-log Primary restart deferred after other units                                                           
2024-05-17T11:25:32.495Z [container-agent] 2024-05-17 11:25:32 INFO juju-log Primary restart deferred after other units                                                           
2024-05-17T11:25:32.712Z [container-agent] 2024-05-17 11:25:32 INFO juju-log Unit workload member-state is online with member-role primary                                        
2024-05-17T11:25:34.637Z [container-agent] 2024-05-17 11:25:34 INFO juju.worker.uniter.operation runhook.go:186 ran "update-status" hook (via hook dispatching script: dispatch)  
2024-05-17T11:26:01.278Z [container-agent] 2024-05-17 11:26:01 INFO juju-log Primary restart deferred after other units                                                           
2024-05-17T11:27:01.600Z [container-agent] 2024-05-17 11:27:01 INFO juju-log Primary restart deferred after other units                                                           
2024-05-17T11:28:01.629Z [container-agent] 2024-05-17 11:28:01 INFO juju-log Primary restart deferred after other units                                                           
2024-05-17T11:29:01.513Z [container-agent] 2024-05-17 11:29:01 INFO juju-log Primary restart deferred after other units

Additional context

This is trying to configure a setting on a mysql app with only 1 unit. In the screenshot, you can see neutron-mysql ok, just because no configuration was changed on it.

github-actions[bot] commented 1 month ago

https://warthogs.atlassian.net/browse/DPE-4411

gboutry commented 3 weeks ago

@paulomach is there news about this bug ? This is preventing sunbeam from being able to scale mysql correctly on single node scenario.

paulomach commented 3 weeks ago

@paulomach is there news about this bug ? This is preventing sunbeam from being able to scale mysql correctly on single node scenario.

@gboutry , yes - there's a fix underway to unblock sunbeam. After that I'll do some deeper refactor on this piece of code to make it more robust.

taurus-forever commented 2 weeks ago

@paulomach consider to add integration CI test for configs changes, you can copy it from postgresql.