search

replicatedhq / kots

KOTS provides the framework, tools and integrations that enable the delivery and management of 3rd-party Kubernetes applications, a.k.a. Kubernetes Off-The-Shelf (KOTS) Software.
https://kots.io
Apache License 2.0
902 stars 90 forks source link

chore(local-dev): add node affinity on the patches for local dev with EC #5002

Closed JGAntunes closed 5 days ago

JGAntunes commented 1 week ago

What this PR does / why we need it:

When running local dev with embedded cluster, running make kotsadm-up-ec will build the local dev images and load them into node0 (by default). If we're running a multi node cluster setup it so happens that the scheduled pods might land in any of the other nodes other than node0 (which won't have the loaded image). This affinity ensures we always schedule the pods on the correct node we set using EC_NODE env var.

Which issue(s) this PR fixes:

NONE

Does this PR require a test?

NONE

Does this PR require a release note?

NONE

Does this PR require documentation?

NONE

sgalsaleh commented 1 week ago

You can use the EC_NODE env var to connect to a different node than node0:

export EC_NODE=node1

This is because you can have multiple clusters and kotsadm should not necessarily always run on node0.

JGAntunes commented 1 week ago

You can use the EC_NODE env var to connect to a different node than node0:

Right but even if you do, for this to work, you need to run make kotsadm-up-ec for each node so that ec_build_and_load loads the image into all the cluster nodes:

Else the deployment will likely fail with pull image backoff errors if the pod lands on a node where the image isn't available.

I thought it would just be easier to pin the deployment to a single node when doing local dev as way to overcome this.

sgalsaleh commented 1 week ago

How about we pin it to the value of EC_NODE then? or node0 if EC_NODE is not specified? Also, need to do this for kurl-proxy as well.

JGAntunes commented 1 week ago

How about we pin it to the value of EC_NODE then? or node0 if EC_NODE is not specified? Also, need to do this for kurl-proxy as well.

Ah got it, yeah that makes sense. I can look into that 👍

JGAntunes commented 1 week ago

Done @sgalsaleh. Let me know what you think.

sgalsaleh commented 1 week ago

Actually, this same patch is used for existing cluster (vanilla KOTS dev env), which I don't believe will work as the node is not named the same way.

JGAntunes commented 1 week ago

Actually, this same patch is used for existing cluster (vanilla KOTS dev env), which I don't believe will work as the node is not named the same way.

Good catch. Went with a different approach, let me know what you think.