MicroK8s provider with channel 1.19 fails to bootstrap

[neoaggelos]

Summary

Using the microk8s provider with 1.19 channel consistently fails while bootstrapping, after enabling storage and dns:

• https://github.com/MiaAltieri/kfserving-operator/runs/4090356818?check_suite_focus=true#step:3:123
• https://github.com/neoaggelos/registry-charm/runs/4090959667?check_suite_focus=true#step:3:123

This Launchpad bug seems relevant, https://bugs.launchpad.net/juju/+bug/1937282.

Things are better in newer microk8s versions (1.22 tested only). The issue seems to be that the apiserver takes a long time before accepting new connections after restarting, so the kubectl call immediately fails.

Proposed fix

Retry the kubectl calls with a back-off, to account for the api server taking longer than expected to come up.

A quickfix can be seen in https://github.com/neoaggelos/actions-operator/blob/19a8cb6692d5c25f8c67eb0f8735a89ee5267baf/dist/bootstrap/index.js#L1597-L1605

which did result in the bootstrapping process succeeding:

• https://github.com/MiaAltieri/kfserving-operator/runs/4091857022?check_suite_focus=true#step:3:125