jetstack / navigator

Managed Database-as-a-Service (DBaaS) on Kubernetes
Apache License 2.0
271 stars 31 forks source link

elasticsearch: reverting the Version field after upgrade has begun will not trigger rollback #310

Open munnerz opened 6 years ago

munnerz commented 6 years ago

This bug effects both 0.1 and master releases of Navigator.

Due to the way the update_version action is implemented, if a user:

Navigator will class the upgrade as 'complete' and not roll back or finish rolling forward the nodes in the pool.

This is because we use an annotation to track the 'current' version of the StatefulSet during an upgrade: https://github.com/jetstack/navigator/blob/d3df0d72fb8113227c037225f54d14fff413a9b8/pkg/controllers/elasticsearch/actions/update_version.go#L51-L67.

This field is not updated until the node pool has been completed updated, meaning if the update is aborted half way through, the annotation will still be set to the old value and so on the next sync iteration, it will detect there are no changes to perform (as currentVersionStr == version str in annotation).

/kind bug