devtron-labs / devtron

Tool integration platform for Kubernetes
https://devtron.ai
Apache License 2.0
4.38k stars 481 forks source link

Bug: Buildx getting timed out waiting for worker nodes to be healthy #4343

Open kamal-devtron opened 11 months ago

kamal-devtron commented 11 months ago

📜 Description

When we are building image with buildx with multi architecture facing timed out issue

Screenshot 2023-12-05 at 11 45 10 AM

👟 Reproduction steps

Enable buildx and in Ci pipeline add both platform arm and amd. It should be using Buildx K8s Driver

👍 Expected behavior

Build should be successful

👎 Actual Behavior

Build is getting timed out

☸ Kubernetes version

1.26

Cloud provider

aws

🌍 Browser

Chrome

🧱 Your Environment

No response

✅ Proposed Solution

No response

👀 Have you spent some time to check if this issue has been raised before?

🏢 Have you read the Code of Conduct?

AB#7195

vikramdevtron commented 4 months ago

@kamal-devtron please confirm still exists or not ?

kamal-devtron commented 4 months ago

It doesn't exists because we are using a cluster autoscaler instead of Karpenter, which resolves the issue

abhibhaw commented 4 months ago

If we use Karpenter again? This will still impact?

kamal-devtron commented 4 months ago

Yes, we have to test it again with same configuration

abhibhaw commented 2 months ago

We will be needing help from Ops team to figure out possible resolutions for this issue, as per our today's discussion assigning it back to @kamal-devtron for some insights around possible solutions. cc: @pawan-mehta-dt @prakarsh-dt