search

dotnet / tye

Tye is a tool that makes developing, testing, and deploying microservices and distributed applications easier. Project Tye includes a local orchestrator to make developing microservices easier and the ability to deploy microservices to Kubernetes with minimal configuration.
MIT License
5.29k stars 520 forks source link

Tye deploy doesn't respect namespace when checking kubectl cluster-info #1544

Open RSoeborg opened 1 year ago

RSoeborg commented 1 year ago

Describe the bug

Some kubernetes providers doesn't allow you to access the "kube system". This is a problem when deploying, since the KubectlDetector class calls kubectl cluster-info.

No matter what namespace is defined for your current context, kubectl cluster-info always tries to retrieve from kube-system unless you directly specify kubectl cluster-info --namespace=my-namespace

The issue is that even if I try to deploy with tye deploy --namespace my-namespace it doesn't use that namespace when it checks if kubectl is connected to a cluster in IsKubectlConnectedToClusterAsync() - so in return I get a Forbidden error back, and since it has a non zero exit code, Tye says that kubectl is not connected to a cluster (even though it is).

Further technical details

Linux running on WSL: cat /etc/os-release

NAME="Ubuntu"
VERSION="20.04 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
RSoeborg commented 1 year ago

Just to add to this issue:

In the ingress validation step, the namespace is set but not used when listing deployments: ListDeploymentForAllNamespacesWithHttpMessagesAsync, giving a 403 Forbidden as well

            // If namespace is null, set it to default
            config.Namespace ??= "default";

            var kubernetes = new Kubernetes(config);

            // Looking for a deployment using a standard label.
            // Note: using a deployment instead of a service - minikube doesn't create a service for the controller.

            try
            {
                var result = await kubernetes.ListDeploymentForAllNamespacesWithHttpMessagesAsync(
                    labelSelector: "app.kubernetes.io/name in (ingress-nginx, nginx-ingress-controller)");
                if (result.Body.Items.Count > 0)
                {
                    foreach (var service in result.Body.Items)
                    {
                        output.WriteInfoLine($"Found existing ingress controller '{service.Metadata.Name}' in namespace '{service.Metadata.NamespaceProperty}'.");
                    }

                    return;
                }
            }

And even with "--Force", the fetch step still occurs and fails (due to the 403) - The Force guard should perhaps be in the top of the file?