search

pulumi / pulumi-kubernetes

A Pulumi resource provider for Kubernetes to manage API resources and workloads in running clusters
https://www.pulumi.com/docs/reference/clouds/kubernetes/
Apache License 2.0
415 stars 115 forks source link

Redpanda chart fails to deploy #3265

Open mattfysh opened 1 month ago

mattfysh commented 1 month ago

What happened?

Using the exact same values, redpanda chart fails to deploy via pulumi but succeeds when deployed via Helm directly

Example

https://github.com/mattfysh/redpanda-deploy-bug

Output of pulumi about

CLI Version 3.136.1 Go Version go1.23.2 Go Compiler gc

Plugins KIND NAME VERSION resource kubernetes 4.18.1 language nodejs 3.136.1-dev.0 resource random 4.16.6

Host OS darwin Version 14.6.1 Arch arm64

This project is written in nodejs: executable='/opt/homebrew/bin/node' version='v22.9.0'

Current Stack: mattfysh/bug/dev

TYPE URN pulumi:pulumi:Stack urn:pulumi:dev::bug::pulumi:pulumi:Stack::bug-dev

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

mattfysh commented 1 month ago

Feel free to close this issue if you're unable to recreate. I suspect this is localized to k8s running in Orbstack on mac

rquitales commented 1 month ago

@mattfysh would you also be able to provide the error message you obtained from the pulumi up operation? In the future, it'd be extremely helpful if you could also provide the error messages. Thanks!

mattfysh commented 1 month ago

hi @rquitales - apologies, there is no error message as the chart deployment hangs. more details here: https://github.com/redpanda-data/helm-charts/issues/1564

When the deployment is stuck, the pods look like this:

redpanda-0                          0/2     Init:2/3   0             1s

The redpanda team found what was causing the issue (insufficient memory limit on the 3rd init container) - so it seems to be an issue with orbstack not managing resources: https://github.com/orbstack/orbstack/issues/1520

However, the reason I opened an issue here is because it was almost always happening when trying to deploy the k8s.helm.v3.Release via the Pulumi CLI. If I deployed the exact same chart with exact same values via the Helm CLI, it would typically deploy without hanging

Reproduction repo is here: https://github.com/mattfysh/redpanda-deploy-bug/tree/main/helm

rquitales commented 1 month ago

Thanks for the additional information here, and the linked issue. When I attempted to repro this issue (using KinD and Docker), I hit another issue related to tsx/compilation, hence why I wanted to see what error you're facing. I believe @blampe was actually able to repro the same error you mentioned on Orbstack. We'll need to investigate this further to see why Helm CLI works as expected so I'll keep this ticket open.