search

Azure / azure-service-operator

Azure Service Operator allows you to create Azure resources using kubectl
https://azure.github.io/azure-service-operator/
MIT License
774 stars 199 forks source link

Add support for manually "progressing the reconciler" style tests #3679

Open matthchr opened 10 months ago

matthchr commented 10 months ago

Describe the current behavior Most of our tests use envtest to run the full reconciler along with go-vcr to record + replay responses.

Describe the improvement We should consider giving ourselves more control by allowing us to also simulate a reconciliation without envtest involved at all. I got this idea from some of the Karpenter tests.

In the ASO context what it would (I think) look like, is a separate test suite that didn't spin up envtest at all, and instead initialized the controller + clients to point to real Azure (w/ record+replay, or possibly w/ mocks), but instead of a feed of events triggering reconciles outside of the tests immediate control, we could manually trigger reconciliation if/when we wanted.

The advantages here are:

  1. We have more ability to easily test states that are not terminal. These types of tests are difficult now in the record+replay envtest framework because the controller never stops sending requests to Azure and eventually "runs out" of requests to replay in replay mode, which will cause the test to be flaky when in replay mode (CI, etc).
  2. We could inject failures into certain Azure client calls and more easily test error control flows. This is technically doable in the record+replay framework too with some work, but error states are often transient and nonterminal (causes issues, see above).

As part of doing this work, it would make sense to also solve #2164 as well.

matthchr commented 3 months ago

We're still interested in doing this, but the testing styles we have now have for the most part been working well.