search

pulumi / pulumi-kubernetes-operator

A Kubernetes Operator that automates the deployment of Pulumi Stacks
Apache License 2.0
222 stars 55 forks source link

Clean up "object has already been modified" warnings #700 #717

Closed EronWright closed 5 days ago

EronWright commented 6 days ago

Proposed changes

This PR optimizes the watches to reduce the likelihood of a race between the cache and the reconciliation loop.

The general idea is to use predicates to filter on the 'edge' events that would allow the reconciler to make forward progress. For example, the workspace reconciler applies a StatefulSet, then waits for it to be ready. Only when the StatefulSet is indeed ready should reconciliation be queued. Before this PR, the watch on StatefulSet was triggering upon any revision change (way too eager).

Overall, a number of optimizations were implemented:

Also, some debug statements were introduced to reason about the revision changes over time, since the "conflicts" are detected based on revision. In particular, we emit a debug statement when reconciliation is triggered by a given watch, to understand 'why'.

Related issues (optional)

Closes #700 Supercedes https://github.com/pulumi/pulumi-kubernetes-operator/pull/679

Example

Here's the result of applying operator/examples/random-yaml/stack.yaml.

Before

controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:26.032Z    ERROR   updating status {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "841b4d5b-029c-4457-a2b7-8c1821475adf", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"}
controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile.func1
controller-manager-6f849dcb6b-4wscw manager     /go/operator/internal/controller/auto/workspace_controller.go:110
controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile
controller-manager-6f849dcb6b-4wscw manager     /go/operator/internal/controller/auto/workspace_controller.go:181
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:114
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:311
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:261
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:222
controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:26.033Z    ERROR   Reconciler error        {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "841b4d5b-029c-4457-a2b7-8c1821475adf", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"}
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:324
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:261
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:222
controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:26.047Z    ERROR   unable to save object status    {"controller": "stack-controller", "namespace": "default", "name": "random-yaml", "reconcileID": "6a99e260-ca2d-4376-b95f-ab490acbe32a", "error": "Operation cannot be fulfilled on stacks.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"}
controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/pulumi.(*StackReconciler).Reconcile.func1
controller-manager-6f849dcb6b-4wscw manager     /go/operator/internal/controller/pulumi/stack_controller.go:445
controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/pulumi.(*StackReconciler).Reconcile
controller-manager-6f849dcb6b-4wscw manager     /go/operator/internal/controller/pulumi/stack_controller.go:741
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:114
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:311
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:261
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:222
controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:26.047Z    ERROR   Reconciler error        {"controller": "stack-controller", "namespace": "default", "name": "random-yaml", "reconcileID": "6a99e260-ca2d-4376-b95f-ab490acbe32a", "error": "Operation cannot be fulfilled on stacks.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"}
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:324
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:261
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:222
controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:26.091Z    ERROR   updating status {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "d397d7d1-c74b-43fe-b4a0-80c3d39ee3ea", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"}
controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile.func1
controller-manager-6f849dcb6b-4wscw manager     /go/operator/internal/controller/auto/workspace_controller.go:110
controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile
controller-manager-6f849dcb6b-4wscw manager     /go/operator/internal/controller/auto/workspace_controller.go:188
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:114
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:311
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:261
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:222
controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:26.091Z    ERROR   Reconciler error        {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "d397d7d1-c74b-43fe-b4a0-80c3d39ee3ea", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"}
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:324
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:261
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:222
controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:26.111Z    ERROR   updating status {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "b94710c8-8165-4937-a99c-a4e065f63788", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"}
controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile.func1
controller-manager-6f849dcb6b-4wscw manager     /go/operator/internal/controller/auto/workspace_controller.go:110
controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile
controller-manager-6f849dcb6b-4wscw manager     /go/operator/internal/controller/auto/workspace_controller.go:188
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:114
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:311
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:261
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:222
controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:26.111Z    ERROR   Reconciler error        {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "b94710c8-8165-4937-a99c-a4e065f63788", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"}
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:324
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:261
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:222
controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:32.082Z    ERROR   updating status {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "7cb6660a-508c-4392-80cd-749fe72def9f", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"}
controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile.func1
controller-manager-6f849dcb6b-4wscw manager     /go/operator/internal/controller/auto/workspace_controller.go:110
controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile
controller-manager-6f849dcb6b-4wscw manager     /go/operator/internal/controller/auto/workspace_controller.go:252
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:114
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:311
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:261
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:222
controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:32.082Z    ERROR   Reconciler error        {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "7cb6660a-508c-4392-80cd-749fe72def9f", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"}
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:324
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:261
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:222
controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:36.114Z    ERROR   updating status {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "480ad23a-2d02-40a6-b4a4-198fb9df01c4", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"}
controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile.func1
controller-manager-6f849dcb6b-4wscw manager     /go/operator/internal/controller/auto/workspace_controller.go:110
controller-manager-6f849dcb6b-4wscw manager github.com/pulumi/pulumi-kubernetes-operator/v2/operator/internal/controller/auto.(*WorkspaceReconciler).Reconcile
controller-manager-6f849dcb6b-4wscw manager     /go/operator/internal/controller/auto/workspace_controller.go:316
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:114
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:311
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:261
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:222
controller-manager-6f849dcb6b-4wscw manager 2024-10-16T01:47:36.114Z    ERROR   Reconciler error        {"controller": "workspace", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "480ad23a-2d02-40a6-b4a4-198fb9df01c4", "error": "Operation cannot be fulfilled on workspaces.auto.pulumi.com \"random-yaml\": the object has been modified; please apply your changes to the latest version and try again"}
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:324
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:261
controller-manager-6f849dcb6b-4wscw manager sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
controller-manager-6f849dcb6b-4wscw manager     /go/pkg/mod/sigs.k8s.io/controller-runtime@v0.18.4/pkg/internal/controller/controller.go:222

After


controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:45.212Z    INFO    Reconciling Stack       {"controller": "stack-controller", "namespace": "default", "name": "random-yaml", "reconcileID": "585499fc-fb33-4505-8594-6a9fd8c810b2", "revision": "893372"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:45.225Z    INFO    KubeAPIWarningLogger    metadata.finalizers: "finalizer.stack.pulumi.com": prefer a domain-qualified finalizer name to avoid accidental conflicts with other finalizer writers
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:45.225Z    INFO    installing watcher for newly seen source kind   {"GroupVersionKind": "source.toolkit.fluxcd.io/v1, Kind=GitRepository"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:45.225Z    INFO    Starting EventSource    {"controller": "stack-controller", "source": "kind source: *unstructured.Unstructured"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:45.238Z    INFO    Reconciling Workspace   {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "d1595cfb-0fe6-46d5-9e9c-2227f334eb7f", "revision": "893374"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:45.238Z    INFO    Applying StatefulSet    {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "d1595cfb-0fe6-46d5-9e9c-2227f334eb7f", "revision": "893374", "hash": "608b7e170ccd95754972cde1178ab426", "source": {"Generation":1,"ForceRequest":"","Git":null,"Flux":{"Url":"http://source-controller.flux-system.svc.cluster.local./gitrepository/default/pulumi-examples/1432c2ce26299516223c0ad228e92294eba4b2c1.tar.gz","Digest":"sha256:44d554df090dcdeb9bae908cd38155c8c93db05f64dc72982027e8e1294af0d3","Dir":"random-yaml/"}}}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:48.609Z    INFO    Reconciling Workspace   {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "5e7b9b87-fab5-4da0-b025-65dadefc097e", "revision": "893380"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:48.609Z    INFO    Applying StatefulSet    {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "5e7b9b87-fab5-4da0-b025-65dadefc097e", "revision": "893380", "hash": "608b7e170ccd95754972cde1178ab426", "source": {"Generation":1,"ForceRequest":"","Git":null,"Flux":{"Url":"http://source-controller.flux-system.svc.cluster.local./gitrepository/default/pulumi-examples/1432c2ce26299516223c0ad228e92294eba4b2c1.tar.gz","Digest":"sha256:44d554df090dcdeb9bae908cd38155c8c93db05f64dc72982027e8e1294af0d3","Dir":"random-yaml/"}}}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:48.719Z    INFO    Connecting to workspace pod     {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "5e7b9b87-fab5-4da0-b025-65dadefc097e", "revision": "893380", "addr": "random-yaml-workspace.default.svc.cluster.local:50051"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:48.724Z    INFO    Connected to workspace pod      {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "5e7b9b87-fab5-4da0-b025-65dadefc097e", "revision": "893380", "addr": "random-yaml-workspace.default.svc.cluster.local:50051"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:48.724Z    INFO    Running pulumi install  {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "5e7b9b87-fab5-4da0-b025-65dadefc097e", "revision": "893380"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:51.509Z    INFO    Creating Pulumi stack(s)        {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "5e7b9b87-fab5-4da0-b025-65dadefc097e", "revision": "893413"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:55.250Z    INFO    workspace pod initialized       {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "5e7b9b87-fab5-4da0-b025-65dadefc097e", "revision": "893422"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:55.250Z    INFO    Ready   {"controller": "workspace-controller", "controllerGroup": "auto.pulumi.com", "controllerKind": "Workspace", "Workspace": {"name":"random-yaml","namespace":"default"}, "namespace": "default", "name": "random-yaml", "reconcileID": "5e7b9b87-fab5-4da0-b025-65dadefc097e", "revision": "893422"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:55.255Z    INFO    Reconciling Stack       {"controller": "stack-controller", "namespace": "default", "name": "random-yaml", "reconcileID": "910a2faa-5b49-4590-a23a-d9ef8cab0023", "revision": "893375"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:55.271Z    INFO    Reconciling Update      {"controller": "update", "controllerGroup": "auto.pulumi.com", "controllerKind": "Update", "Update": {"name":"random-yaml-b7kng2l5","namespace":"default"}, "namespace": "default", "name": "random-yaml-b7kng2l5", "reconcileID": "5b9ced73-989f-46fa-a485-746686519832"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:55.271Z    INFO    Updating the status     {"controller": "update", "controllerGroup": "auto.pulumi.com", "controllerKind": "Update", "Update": {"name":"random-yaml-b7kng2l5","namespace":"default"}, "namespace": "default", "name": "random-yaml-b7kng2l5", "reconcileID": "5b9ced73-989f-46fa-a485-746686519832"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:55.275Z    INFO    Connecting      {"controller": "update", "controllerGroup": "auto.pulumi.com", "controllerKind": "Update", "Update": {"name":"random-yaml-b7kng2l5","namespace":"default"}, "namespace": "default", "name": "random-yaml-b7kng2l5", "reconcileID": "5b9ced73-989f-46fa-a485-746686519832", "addr": "random-yaml-workspace.default.svc.cluster.local:50051"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:55.278Z    INFO    Selecting the stack     {"controller": "update", "controllerGroup": "auto.pulumi.com", "controllerKind": "Update", "Update": {"name":"random-yaml-b7kng2l5","namespace":"default"}, "namespace": "default", "name": "random-yaml-b7kng2l5", "reconcileID": "5b9ced73-989f-46fa-a485-746686519832", "stackName": "dev"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:57.333Z    INFO    Applying the update     {"controller": "update", "controllerGroup": "auto.pulumi.com", "controllerKind": "Update", "Update": {"name":"random-yaml-b7kng2l5","namespace":"default"}, "namespace": "default", "name": "random-yaml-b7kng2l5", "reconcileID": "5b9ced73-989f-46fa-a485-746686519832", "type": "up"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:01:57.334Z    INFO    Executing update operation      {"controller": "update", "controllerGroup": "auto.pulumi.com", "controllerKind": "Update", "Update": {"name":"random-yaml-b7kng2l5","namespace":"default"}, "namespace": "default", "name": "random-yaml-b7kng2l5", "reconcileID": "5b9ced73-989f-46fa-a485-746686519832", "request": "message:\"Stack Update (up)\"  target_dependents:false  refresh:true"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:02:05.421Z    INFO    Update complete {"controller": "update", "controllerGroup": "auto.pulumi.com", "controllerKind": "Update", "Update": {"name":"random-yaml-b7kng2l5","namespace":"default"}, "namespace": "default", "name": "random-yaml-b7kng2l5", "reconcileID": "5b9ced73-989f-46fa-a485-746686519832", "result": "stdout:\"Updating (dev)\\n\\nView Live: https://app.pulumi.com/eron-pulumi-corp/random/dev/updates/4536\\n\\n+ pulumi:pulumi:Stack: (create)\\n    [urn=urn:pulumi:dev::random::pulumi:pulumi:Stack::random-dev]\\n    + random:index/randomPassword:RandomPassword: (create)\\n        [urn=urn:pulumi:dev::random::random:index/randomPassword:RandomPassword::randomPassword]\\n        length         : 16\\n        overrideSpecial: \\\"_%@\\\"\\n        special        : true\\n        --outputs:--\\n        bcryptHash     : [secret]\\n        id             : \\\"none\\\"\\n        lower          : true\\n        minLower       : 0\\n        minNumeric     : 0\\n        minSpecial     : 0\\n        minUpper       : 0\\n        number         : true\\n        numeric        : true\\n        result         : [secret]\\n        upper          : true\\n    --outputs:--\\n    password: [secret]\\nResources:\\n    + 2 created\\n\\nDuration: 3s\\n\"  summary:{start_time:{seconds:1729044119}  end_time:{seconds:1729044122}  result:\"succeeded\"  message:\"\\\"Stack Update (up)\\\"\"}  permalink:\"https://app.pulumi.com/eron-pulumi-corp/random/dev/updates/4536\"  outputs:{key:\"password\"  value:{value:\"\\\"fAMcmXI2DfW7KCey\\\"\"  secret:true}}"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:02:05.438Z    INFO    Reconciling Stack       {"controller": "stack-controller", "namespace": "default", "name": "random-yaml", "reconcileID": "a100c4f3-c5b9-4894-8644-fabc1f71d008", "revision": "893437"}
controller-manager-778bdbdfbb-cdrpb manager 2024-10-16T02:02:05.546Z    INFO    Commit hash unchanged. Will wait for Source update or resync.   {"controller": "stack-controller", "namespace": "default", "name": "random-yaml", "reconcileID": "a100c4f3-c5b9-4894-8644-fabc1f71d008", "revision": "893437"}
codecov[bot] commented 6 days ago

Codecov Report

Attention: Patch coverage is 16.66667% with 135 lines in your changes missing coverage. Please review.

Project coverage is 51.20%. Comparing base (7883699) to head (0c4e8cb). Report is 1 commits behind head on v2.

Files with missing lines Patch % Lines
...tor/internal/controller/pulumi/stack_controller.go 23.33% 46 Missing :warning:
...ator/internal/controller/auto/update_controller.go 2.38% 41 Missing :warning:
...r/internal/controller/auto/workspace_controller.go 16.66% 19 Missing and 1 partial :warning:
operator/internal/controller/pulumi/flux.go 33.33% 13 Missing and 3 partials :warning:
operator/internal/controller/auto/utils.go 0.00% 12 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## v2 #717 +/- ## ========================================== - Coverage 53.00% 51.20% -1.80% ========================================== Files 29 29 Lines 3081 3199 +118 ========================================== + Hits 1633 1638 +5 - Misses 1267 1379 +112 - Partials 181 182 +1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.