Kudo init --upgrade should wait on deletion before proceeding with upgrade

[alenkacz]

What happened: Currently, when uninstalling stuff as a first step of upgrade, we use foreground delete propagation https://github.com/kudobuilder/kudo/blob/5c68af2007d6e165d3a9627f12aa9584dc16782c/pkg/kudoctl/kudoinit/manager/manager.go#L73 and we falsely assumed that's blocking until it gets deleted which is not true. API server returns response when object is marked to be deleted but then it could take a substantial amount of time until object gets deleted (depending on the finalizers).

When deleting something as part of upgrade, we should always "block" and wait until really deleted (probably with some timeout) and not proceed with upgrade until that's done.

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?:

Environment:

• Kubernetes version (use kubectl version):
• Kudo version (use kubectl kudo version):
• Operator:
• operatorVersion:
• Cloud provider or hardware configuration:
• OS (e.g. from /etc/os-release):
• Kernel (e.g. uname -a):
• Install tools:
• Others:

[kensipe]

closely related: https://github.com/kudobuilder/kudo/issues/1631