search

numaproj / numaplane

Control Plane for Numaproj
Apache License 2.0
8 stars 4 forks source link

fix: checking Pipeline conditions for PPND, plus don't pause failed pipelines #238

Closed juliev0 closed 2 weeks ago

juliev0 commented 2 weeks ago

Fixes #230

Modifications

Modification 1: Don't pause "Failed" pipelines:

  1. I don't pause pipelines if they're "Failed"
  2. I can update the pipeline spec without waiting for a "Failed" pipeline to be "Paused"
  3. I can update NumaflowController and ISBService specs if Pipelines are either "Paused" or "Failed"
Verification
  1. unit test checks ISBService (same code is used for Numaflow Controller so presumably that is being verified at the same time)
  2. manually tested this sequence: create bad pipeline which failed validation; attempted to update failed pipeline with a change which didn't fix the problem; updated failed pipeline by fixing the validation error
  3. manually tested updating ISBServiceRO and NCRO while pipeline was failed

Modification 2: Fix Pipeline Condition checking for PPND

Was accidentally checking the PipelineRollout.Conditions rather than Pipeline.Conditions when waiting for the Pipeline to be fully reconciled.

Verification

I observed in the log that at the time we resumed the Pipeline all of the Child Conditions were in a healthy state. I also observed that the daemon was up and running.

Modification 3: e2e test

Have incorporated latest Numaflow release into e2e test. Have increased suite timeout and added a comment in case we need to increase further in the future. Have fixed the Makefile which was setting environment variable for the test related to bringing up a separate kubernetes environment, which we aren't using.

juliev0 commented 2 weeks ago

I will do some manual testing of pipeline changes before marking this "Ready for Review"