How to deploy a script/application on GKE using deployment manager, right after the cluster creation is done.

[Priyankasaggu11929]

I want to make a deployment of the application/product while the cluster creation only.

I see AWS has AWS lambda functions for the purpose. And similarily GCP provide Cloud Functions as a serverless service, but I don't see any relavant example to start in the above use-case direction.

What should be the solution for the same?

[ocsig]

I wouls argue that the management of GKE services are outside the scope of deployment manager. You are looking for a post deployment step. Yes, this can be a CloudFunction or a CloudBuild step, just you need to be sure that the GKE cluster is already up and running. I would add this as a next step in your CICD pipeline.

[Priyankasaggu11929]

@ocsig

I would argue that the management of GKE services are outside the scope of deployment manager. You are looking for a post deployment step.

yes, I understood your point.

But the issue is we don't have a CI/CD pipeline on the client enterprise end.

So, the use-case is to add this deployment step during the infrastructure deployment itself.

Something like, we want to achieve a one-click product deployment.

[ocsig]

Try to add a Cloudbuild in your DM deployment after the cluster deployment ( with a dependency) which runs kubectl commands and has the required source files. You may need to set presteps for waiting the cluster to be ready.

[Priyankasaggu11929]

Thank you, I'll try building the solution with Cloudbuild.

[Priyankasaggu11929]

@ocsig

[UPDATES]

I used this template as a reference here.

So, my template basically looks like this:

resources:
- name: build-something
  action: gcp-types/cloudbuild-v1:cloudbuild.projects.builds.create
  metadata:
    runtimePolicy:
    - UPDATE_ALWAYS
  properties:
    steps:
    - name: 'gcr.io/cloud-builders/kubectl'
      args:
      - get
      - pods
      env:
      - 'CLOUDSDK_COMPUTE_ZONE=us-west1-a'
      - 'CLOUDSDK_CONTAINER_CLUSTER=demo-cluster'

It does work in a public gke cluster. But throws unable to connect to the server error whenever run on a private gke cluster.

In general, to access a gke private cluster, I add required ipcidr range to Master Authorized Networks.

But here, If I'm not wrong, cloud build runs it in an ephermeral cloud shell instance, so I can't somehow know what the corresponding IP is, in order to allow it.

Below is the failed logs:

starting build "64c64c1c-9cb5-4dce-bb79-eb5d9e01e634"

FETCHSOURCE
BUILD
Already have image (with digest): gcr.io/cloud-builders/kubectl
Running: gcloud container clusters get-credentials --project="project-name" --zone="us-west1-a" "demo-cluster"
Fetching cluster endpoint and auth data.
kubeconfig entry generated for demo-cluster.
Running: kubectl get pods
Unable to connect to the server: dial tcp {ip}:443: i/o timeout
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/kubectl" failed: step exited with non-zero status: 1

So, I even tried adding Kubernetes Engine Admin permission/role along with Kubernetes Engine Developer to the cloudbuild service account. It still didn't work.

[Priyankasaggu11929]

@ocsig

There is another error I receive generally whenever I try to kubectl apply on a yaml file inside a google cloud shell on the same yaml file.

but doesn't happen whenever I run the same command from a local machine's terminal.

In this case, when I tried to run kubectl apply -f deployment.yaml from the cloudbuild DM template, (again I'm assuming it ran inside a cloud shell),

so, I got the error error: error parsing https://storage.cloud.google.com/{bucket-name}/deployment.yaml: error converting YAML to JSON: yaml: line 11: mapping values are not allowed in this context.

Logs below:

starting build "ffaa37ba-aa65-4be3-94eb-db29b63561dc"

FETCHSOURCE
BUILD
Already have image (with digest): gcr.io/cloud-builders/kubectl
Running: gcloud container clusters get-credentials --project="{project-id}" --zone="us-west1-a" "demo-cluster"
Fetching cluster endpoint and auth data.
kubeconfig entry generated for demo-cluster.
Running: kubectl apply -f https://storage.cloud.google.com/{bucket-name}/deployment.yaml
error: error parsing https://storage.cloud.google.com/{bucket-name}/deployment.yaml: error converting YAML to JSON: yaml: line 11: mapping values are not allowed in this context
ERROR
ERROR: build step 0 "gcr.io/cloud-builders/kubectl" failed: step exited with non-zero status: 1

Whenever I upload a file to gcs bucket, by some means the formatting gets changed. So, it throws the above error.

Do you think, I can somehow ensure the indentation/formatting here?

[thebeardedpapi]

@Priyankasaggu11929 - ever figure out how to deploy into your private GKE cluster?

The examples in the repo don't seem to work for me, tried triggering cloud build from deployment manager, got the same timeout error :disappointed: