search

nais / naiserator

Naiserator creates a full set of Kubernetes application infrastructure based on a single application spec.
MIT License
43 stars 12 forks source link

Installation on cluster #456

Closed vpipkt closed 1 year ago

vpipkt commented 1 year ago

I have been taking a look at nais and using the CLI was able to generate a nais.io/v1alpha1 Application config / template for my application. Now time to deploy it on my kubernetes cluster.

I followed these instructions: https://github.com/nais/naiserator#deployment

And I get a Deployment and a pod with failing readiness checks. I assume this failing pod is supposed to be the Controller for the Application type. What I expected but don't get is a nais.io/v1alpha1 Application API resource on my cluster.

The event I'm getting on the naiserator pod is Warning Unhealthy Pod/naiserator-7c87ff67f8-d9k95 Readiness probe failed: Get "http://10.244.1.3:8085/readyz": dial tcp 10.244.1.3:8085: connect: connection refused That IP address is the pod's own address.

In other pods on my cluster I'm able to have pods pass similar readiness probes.

Any advice on troubleshooting the pod? Like where in that container to find relevant logs? Are there configurations or arguments I might need to change in the Deployment spec? Any experience with similar issues?

Thanks for taking a look.

thokra-nav commented 1 year ago

Hi! We have some troubleshooting steps documented here: https://docs.nais.io/deployment/troubleshooting/

Depending on how naiserator is configured, a pod might have more than one container running within it. I would describe it with kubectl describe pod -n [namespace] [podname] and see which container is failing.

If it's a container with the same name as the application, ensure that:

You can also tail the logs using kubectl logs [pod] [-c container] (You only need -c if you want to check the logs for a container that's not your app)

sechmann commented 1 year ago

What I expected but don't get is a nais.io/v1alpha1 Application API resource on my cluster.

The CRD itself it built by https://github.com/nais/liberator, and can be found here: https://github.com/nais/liberator/tree/main/charts/templates

mortenlj commented 1 year ago

As the previous comments indicates, it's slightly unclear exactly what you are trying to do. :slightly_smiling_face:

If you are trying to install nais in a cluster of your own, there aren't really any good documentation, and the information in the README in this repo is probably outdated/incorrect.

First of, nais is a big ball of interconnected services and infrastructure, and as of now there isn't any clear guide for how to install and operate it on your own. I'll try to draw up some of the important aspects here, see if that can be enough to get you started.

The bare minimum needed in a cluster would be the CRDs and Naiserator, but that alone would probably only give you a very rudimentary "nais cluster". CRDs are installed with the chart referenced by @sechmann in the above comment. Naiserator is probably best to install using the chart in https://github.com/nais/naiserator/tree/master/charts/naiserator.

You can see there are quite a few values to adjust in those charts, with very little documentation on what each does. We don't have any better option than trying them out or tracing their use in the code to understand them more.

The next step you might want to install is the deploy system (https://github.com/nais/deploy/tree/master/charts), which consists of a hookd instance available from anywhere you would like to deploy from (Github, local machine, etc), and a deployd instance running in the cluster you wish to deploy to. Deployd in the cluster needs to be able to reach hookd (the cluster doesn't need to be directly on the internet, as long as deployd can call out to hookd). Deploying is done using the deploy cli (or the deploy github actions), which takes some inputs and sends them to hookd, which forwards to deployd in the selected cluster, which applies the manifests in the cluster.

If you stop there, you have the basics needed for deploying applications, but none of the integrations to services outside the cluster. Other components you might want to look at are listed below, although some of these may contain NAV specific code that wouldn't work outside our clusters. Documentation on each is also limited:

This is a very quick, high-level introduction, but for now it's all we have :sweat_smile:

vpipkt commented 1 year ago

Thanks for all the feedback everyone!

@thokra-nav I took a look at the troubleshooting docs and that seems more oriented to trouble with deployment of an app, but my problem is configuring my k8s cluster to use naiserator. Then later I will deploy apps with naiserator but not there yet.

@sechmann I'll take a look at the liberator repo as well.

Of course this from @mortenlj is really important to learn.

If you are trying to install nais in a cluster of your own, there aren't really any good documentation, and the information in the README in this repo is probably outdated/incorrect.

But many many thanks for the summary there. I will try to let you all know if I have success with the instructions in the comment there.

Just one more naive question before I dive any deeper into deploy. Suppose I get to the state of a rudimentary nais cluster (CRD+naiserator). From there is there a need for the deploy system? Could I not just deploy Application resources using vanilla k8s tools, e.g. kubectl create or helm?

mortenlj commented 1 year ago

From there is there a need for the deploy system? Could I not just deploy Application resources using vanilla k8s tools, e.g. kubectl create or helm?

You don't need the deploy system, you can just use the vanilla k8s tools if you want to do that. The deploy system is designed to make it easier to support deploy to multiple clusters from outside a secure network. If you don't need that, you don't need the deploy system.

I have taken this issue as a kick off to start on some proper documentation for installing nais, although so far it only contains what I posted here. In the future we might add more details to it. It can be found in our handbook.

I'm going to close this issue now, but feel free to reach out if you need more help. We are also available for chatting in the #nais channel on Kubernetes Slack.