Azure / azure-service-operator

Azure Service Operator allows you to create Azure resources using kubectl
https://azure.github.io/azure-service-operator/
MIT License
738 stars 194 forks source link

Reduce the noise in our golden tests #2150

Open theunrepentantgeek opened 2 years ago

theunrepentantgeek commented 2 years ago

Describe the current behavior

Many of our golden tests (the ones driven by test data under v2\tools\generator\internal\codegen\testdata) are checking for the structure of the final data types, but also contain a lot of other generated code that's simply noise.

Describe the improvement

Reduce the amount of generated code so that the data structures become the focus.

Additional context

We can't just filter out all functions - tried that, it suppressed too much.

We also can't just pull out some of the pipeline stages. For one, they're often prerequisites of other stages that we do want to test. For another, a lot of our stages are ONLY tested by these golden tests (they don't have isolated unit tests of their own). The more our test pipeline resembles the actual one we use for code generation the better, because we rule out unexpected interactions between the stages.

theunrepentantgeek commented 2 years ago

Our test package already includes a fluent style API for doing assertions:

test.AssertPackagesGenerateExpectedCode(
    t, 
    finalState.definitions, 
    test.DiffWithTypes(initialState.Definitions()))

We might add more options to allow removing specific functions and/or interface implementations before generating the golden files:

test.AssertPackagesGenerateExpectedCode(
    t, 
    finalState.definitions,
    test.RemoveAll(&PropertyAssignmentFunction{}),
    test.RemoveAll(&PivotConversionFunction{}),
    test.RemoveAll(&OriginalGVKFunction{}))

These would work by pruning any functions (or interface implementations) that match the specified types.

matthchr commented 2 years ago

This is something we're still interested in doing, but its not a high priority right now.

matthchr commented 1 year ago

Still interested in this, but it's not a high priority currently

matthchr commented 5 months ago

This is still something we're interested in, and related to getting rid of gojsonschema too (#1592)

theunrepentantgeek commented 1 week ago

I'm still interested in doing this.