Pipeline never paused

[juliev0]

Describe the bug The Numaplane e2e test passes most of the time but I want it to pass 100% of the time. This Issue describes a case.

This is the sequence of steps it had executed up until when the Pipeline never got paused:

• create isbsvc, wait for it and StatefulSet to be reconciled
• once it's ready, create pipeline
• wait for it to be "Running"
• pause pipeline
• wait for it to be "Paused"
• update spec, keeping desiredPhase=Paused (in this case it makes a minor spec change to source.generator.rpu)
• wait for it to be reconciled
• run pipeline
• wait for it be "Running"
• repeat steps above starting from "pause pipeline"
• then "pause pipeline" one more time <-- this is where it got stuck

Pipeline spec:

pipelineSpec        = numaflowv1.PipelineSpec{
        InterStepBufferServiceName: isbServiceRolloutName,
        Vertices: []numaflowv1.AbstractVertex{
            {
                Name: "in",
                Source: &numaflowv1.Source{
                    Generator: &numaflowv1.GeneratorSource{
                        RPU:      &pipelineSpecSourceRPU,
                        Duration: &pipelineSpecSourceDuration,
                    },
                },
                Scale: numaflowv1.Scale{Min: &numVertices, Max: &numVertices, ZeroReplicaSleepSeconds: &zeroReplicaSleepSec},
            },
            {
                Name: "out",
                Sink: &numaflowv1.Sink{
                    AbstractSink: numaflowv1.AbstractSink{
                        Log: &numaflowv1.Log{},
                    },
                },
                Scale: numaflowv1.Scale{Min: &numVertices, Max: &numVertices, ZeroReplicaSleepSeconds: &zeroReplicaSleepSec},
            },
        },
        Edges: []numaflowv1.Edge{
            {
                From: "in",
                To:   "out",
            },
        },
    }

This zip file contains the following:

• full Numaflow log file, to which I added a line "NUMAPLANE LOG" with corresponding time stamp each time Numaplane performs some action (creating isbsvc, creating pipeline, updating pipeline)
• result of kubectl get pipeline -o yaml at the time it was remaining pausing

This is the full numaplane log.

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We often sort issues this way to know what to prioritize.

For quick help and support, join our slack channel.

[juliev0]

looks like github closed this prematurely when you'd only written "partially fixes" in the PR, so reopening