search

gimlet-io / gimlet

Deployment tool built on Kubernetes to make the deploy, preview and rollback workflows accessible to everyone.
https://gimlet.io
Other
333 stars 27 forks source link

Agent startup crashes on a null pointer for artifact revision #815

Closed sukkola closed 3 months ago

sukkola commented 9 months ago

Freshly initialised installation is unable to start the gimlet-agent

time="2024-02-03T11:18:17Z" level=warning msg="could not load .env file, relying on env vars" │ │ time="2024-02-03T11:18:17Z" level=info msg="Initializing lab-dev kubeEnv in cluster scope" file="[agent.go:84]" │ │ time="2024-02-03T11:18:17Z" level=info msg="Starting ingress controller" file="[controller.go:224]" │ │ time="2024-02-03T11:18:17Z" level=info msg="Starting pod controller" file="[controller.go:224]" │ │ time="2024-02-03T11:18:17Z" level=info msg="Starting deployment controller" file="[controller.go:224]" │ │ time="2024-02-03T11:18:17Z" level=info msg="Starting gitrepositories.source.toolkit.fluxcd.io controller" file="[controller.go:224]" │ │ time="2024-02-03T11:18:17Z" level=info msg="Starting helmreleases.helm.toolkit.fluxcd.io controller" file="[controller.go:224]" │ │ time="2024-02-03T11:18:17Z" level=info msg="Starting kustomizations.kustomize.toolkit.fluxcd.io controller" file="[controller.go:224]" │ │ time="2024-02-03T11:18:17Z" level=info msg=Initialized file="[agent.go:154]" │ │ time="2024-02-03T11:18:17Z" level=info msg="Connected ws" file="[agent.go:575]" │ │ W0203 11:18:17.227020 1 warnings.go:70] v2beta1 HelmRelease is deprecated, upgrade to v2beta2 │ │ time="2024-02-03T11:18:17Z" level=info msg="Connected to Gimlet" file="[agent.go:221]" │ │ W0203 11:18:17.307868 1 warnings.go:70] v2beta1 HelmRelease is deprecated, upgrade to v2beta2 │ │ panic: runtime error: invalid memory address or nil pointer dereference │ │ [signal SIGSEGV: segmentation violation code=0x1 addr=0x20 pc=0x1533ff4] │ goroutine 149 [running]: │ │ github.com/gimlet-io/gimlet-cli/pkg/agent.asGitRepository({0x4000989830}) │ │ /home/runner/work/gimlet/gimlet/pkg/agent/kube.go:293 +0x144 │ │ github.com/gimlet-io/gimlet-cli/pkg/agent.(*KubeEnv).GitRepositories(0x4000539270?) │ │ /home/runner/work/gimlet/gimlet/pkg/agent/kube.go:210 +0x138 │ │ github.com/gimlet-io/gimlet-cli/pkg/agent.SendFluxState(0x4000539270, {0x4000058005, 0x2c}, {0x400005400a, 0x74}) │ │ /home/runner/work/gimlet/gimlet/pkg/agent/gitRepositoryController.go:41 +0x40 │ │ main.sendFluxState(0x0?, {0x4000058005?, 0x0?}, {0x400005400a?, 0x1c?}) │ │ /home/runner/work/gimlet/gimlet/cmd/agent/agent.go:333 +0x24 │ │ created by main.serverCommunication │ │ /home/runner/work/gimlet/gimlet/cmd/agent/agent.go:224 +0x2c8

laszlocph commented 9 months ago

Crashes here: https://github.com/gimlet-io/gimlet/blob/main/pkg/agent/kube.go#L293

The agent wrongly assumes that there is always a fetched revision on the GitRepository source.

I believe there is a workaround: fixing the GitRepository to fetch the revision will stop the agent from crashing.

We are working on #806 which deprecate all these codepaths in the agent. We are going to consider this case and make the agent more robust.