search

numaproj / numaplane

Control Plane for Numaproj
Apache License 2.0
4 stars 4 forks source link

Don't cache children of Numaflow Controller Rollout twice #6

Open juliev0 opened 1 month ago

juliev0 commented 1 month ago

Currently, we are calling Watch() on Resources in NumaflowControllerRollout here but we also have these same resources being cached by the gitops-engine.

There is a call to clusterCache.OnResourceUpdated() in sync/cache.go. This uses a callback. We would need to make sure that we safely call the same reconciliation code that is being used by Reconcile(), such that only one NumaflowControllerRollout is being processed at a time. Thus, we would probably need to use a workqueue.RateLimitingInterface where both Reconcile() and this callback add to that queue (similar to what we're planning for PipelineRollout).

juliev0 commented 1 week ago

FYI @afugazzotto - there was an issue here before, but I updated it to better describe the issue