search

hashicorp / terraform

Terraform enables you to safely and predictably create, change, and improve infrastructure. It is a source-available tool that codifies APIs into declarative configuration files that can be shared amongst team members, treated as code, edited, reviewed, and versioned.
https://www.terraform.io/
Other
42.31k stars 9.49k forks source link

Performance slow down between 1.8.5 and 1.9.2? #35528

Open alexhung opened 1 month ago

alexhung commented 1 month ago

Terraform Version

1.8.5 and 1.9.2

Terraform Configuration Files

N/A

Debug Output

N/A

Expected Behavior

Acceptance tests duration stays roughly the same.

Actual Behavior

Acceptance tests duration increased by 2x.

Steps to Reproduce

Execute my GitHub workflow acceptance test

Additional Context

Since Terraform 1.9.x, I noticed in our test suite that it nows executes around 2x slower.

With 1.8.5, it took around 19 min to complete all the acceptance testing:

a46783cbddee6147b64057f41d02da7502ac18b5

With 1.9.2, now it took roughly twice as long:

2ee0613a387ae808ed5ac484f8b022283a861a10

GitHub workflow: https://github.com/jfrog/terraform-provider-artifactory/actions/workflows/acceptance-tests.yml

The pre-slowdown test run: https://github.com/jfrog/terraform-provider-artifactory/actions/runs/9701273096 PR: https://github.com/jfrog/terraform-provider-artifactory/pull/1008/files using Terraform 1.8.5

The next run: https://github.com/jfrog/terraform-provider-artifactory/actions/runs/9881625510 PR: https://github.com/jfrog/terraform-provider-artifactory/pull/1019/files using Terraform 1.9.2

References

No response

jbardin commented 1 month ago

Hi @alexhung,

Thanks for reporting this. I'm looking into it now, but not coming up with any timing differences through our usual collection of provider tests. Would it be possible to pick out a test scenario which shows a timing regression, and configure the same changes to be executed manually with terraform? That would eliminate any extra variables besides the core version, and make tracing any difference easier.

Thanks!

alexhung commented 1 month ago

@jbardin I tried to reproduce this locally (not on GitHub workflow) and have been unable to find any differences between Terraform 1.8.5, 1.9.3, or even OpenTofu 1.8.0.

Same subset of acceptance tests. This set takes around 220 seconds to complete locally. All three variations take about the same amount of time:

And yet, the GitHub workflow shows 2x differences. 😕

liamcervante commented 1 month ago

This might be a result of the security fix in https://github.com/hashicorp/terraform/releases/tag/v1.9.1. @alexhung, are you including timing of terraform init in your local runs and / or in the acceptance tests?

alexhung commented 1 month ago

@liamcervante The timings data above are from acceptance tests.