search

replicatedhq / kots

KOTS provides the framework, tools and integrations that enable the delivery and management of 3rd-party Kubernetes applications, a.k.a. Kubernetes Off-The-Shelf (KOTS) Software.
https://kots.io
Apache License 2.0
898 stars 89 forks source link

When installed to non-"default" namespace, Preflight pods still created in "default" #1141

Open MikaelSmith opened 4 years ago

MikaelSmith commented 4 years ago

I've installed KOTS to a namespace other than default. I have preflight checks that run pods where we've left the namespace property blank as it shouldn't matter what namespace they run. And my application is configured with requireMinimalRBACPrivileges: true.

https://troubleshoot.sh/docs/collect/run/#namespace-optional claims that an empty namespace

will assume the "current" namespace that kubectl context is set to

However when trying to install my application, preflight checks won't run because

cannot collect run/object-store-credentials-valid: action "create" is not allowed on resource "Pod" in the "default" namespace
cannot collect run/ingress-certs-valid: action "create" is not allowed on resource "Pod" in the "default" namespace
cannot collect run/ingress-hosts-valid: action "create" is not allowed on resource "Pod" in the "default" namespace
cannot collect cluster-resources: action "list" is not allowed on resource "Namespace" at the cluster scope
cannot collect cluster-resources: action "list" is not allowed on resource "Node" at the cluster scope
cannot collect cluster-resources: action "list" is not allowed on resource "CustomResourceDefinition" at the cluster scope
cannot collect cluster-resources: action "list" is not allowed on resource "StorageClasses" at the cluster scope

The cluster-resources failures make sense because no RBAC is setup at cluster scope (it'd be nice to have a flag to override this, because it's not really application-specific so much as specific to the install environment.

KOTS is trying to run the "Run Pods" preflight collectors in the default namespace, which it doesn't have permission to do. But conceptually KOTS's "current namespace" is the one it was installed to, so it would make sense for that to be the one it executes "Run Pods" in.

"Run Pods" collectors without an explicit namespace should work with a KOTS install where requireMinimalRBACPrivileges is set to true.

MikaelSmith commented 3 years ago

It seems like https://github.com/replicatedhq/kots/blob/v1.24.0/kotsadm/pkg/preflight/execute.go#L44 should be set to the POD_NAMESPACE, not left blank. Although I'm not sure the troubleshoot docs are right, it seems to use default if no specific namespace is set.

dexhorthy commented 3 years ago

@MikaelSmith it definitely seems like there's room for improvement here, and I agree that if namespace is blank, the pods should run wherever KOTS is installed.

As a temporary workaround, have you tried using {{repl Namespace }} for the exec preflights?

MikaelSmith commented 3 years ago

Ah yeah, forgot they're templated for a minute. I can use that as a workaround.

tamarahenson commented 3 years ago

Update: I have logged internal request #28603 for this issue.