Investigate why VCR cassettes are invalidated when we update the TF version used in VCR testing automation

slevenick commented 2 months ago

Community Note

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
If you are interested in working on this issue or have submitted a pull request, please leave a comment.
If an issue is assigned to a user, that user is claiming responsibility for the issue.
Customers working with a Google Technical Account Manager or Customer Engineer can ask them to reach out internally to expedite investigation and resolution of this issue.

Description

When the version in our CI tests gets changed it triggers re-recording of every test, which tends to cause problems. ~We may be able to ignore this header specifically during cassette matching~

We thought this could be due to the User-Agent changing as a result of the TF version change, but this has been shown as incorrect.

When the TF version is next updated we should be prepared for VCR cassettes to be invalidated and try to learn why it happens, assuming it does.

New or Affected Resource(s)

all

Potential Terraform Configuration

References

No response

SarahFrench commented 2 months ago

I'm interested in taking this on

SarahFrench commented 2 months ago

I'm a bit confused, as I'm pretty sure I remember upgrading the Terraform version caused an issue with VCR cassettes. However the tests I've added in the PR above suggest the User-Agent header has no impact on matching.

slevenick commented 2 months ago

That checks out with my understanding of the existing matcher. I don't believe we compare on any part of the header currently. I specifically wrote a custom matcher because we need to avoid matching on some header fields (auth!)

I'm not sure why we saw the TF version cause issues with cassettes in the past though, that's the worrying part

SarahFrench commented 1 month ago

I'm going to repurpose this issue, now we know that the user-agent doesn't impact VCR cassettes, to instead be about monitoring the upcoming bump in the Terraform version used on PRs.

SarahFrench commented 4 days ago

@BBBmau - I've reassigned this to you - when the upcoming TF version bump on PRs happens please ask the On Call engineer to help address any invalidated cassettes and also help with understanding why cassettes are invalidated (assuming that the same is seen as last time).

hashicorp / terraform-provider-google