search

hashicorp / terraform-plugin-testing

Module for testing Terraform providers
Mozilla Public License 2.0
44 stars 11 forks source link

tfversion: Finish prerelease version semantic equality handling #317

Closed bflad closed 3 months ago

bflad commented 3 months ago

Closes #303 Reference: https://github.com/hashicorp/terraform-plugin-testing/pull/308 Reference: https://github.com/hashicorp/terraform-plugin-testing/pull/316

These changes fully update the rest of the tfversion built-in checks for treating prereleases as semantically equal to the patch version. It also includes a fix for RequireBelow to handle the exactly equal condition. Previously, RequireBelow when given 1.8.0 would still run on 1.8.0 (and prereleases variants).

Prior to logic changes:

=== RUN   Test_RequireAbove
--- PASS: Test_RequireAbove (3.01s)
=== RUN   Test_RequireAbove_Error
--- PASS: Test_RequireAbove_Error (3.24s)
=== RUN   Test_RequireAbove_Prerelease_EqualCoreVersion
    require_above_test.go:77: expected Terraform CLI version above 1.8.0 but detected version is 1.8.0-rc1
--- FAIL: Test_RequireAbove_Prerelease_EqualCoreVersion (3.97s)
=== RUN   Test_RequireAbove_Prerelease_HigherCoreVersion
--- PASS: Test_RequireAbove_Prerelease_HigherCoreVersion (3.17s)
=== RUN   Test_RequireAbove_Prerelease_HigherPrerelease
--- PASS: Test_RequireAbove_Prerelease_HigherPrerelease (3.03s)
=== RUN   Test_RequireAbove_Prerelease_LowerCoreVersion
--- PASS: Test_RequireAbove_Prerelease_LowerCoreVersion (3.13s)
=== RUN   Test_RequireAbove_Prerelease_LowerPrerelease
--- PASS: Test_RequireAbove_Prerelease_LowerPrerelease (3.14s)

=== RUN   Test_RequireBelow
--- PASS: Test_RequireBelow (2.94s)
=== RUN   Test_RequireBelow_Error
--- PASS: Test_RequireBelow_Error (2.59s)
=== RUN   Test_RequireBelow_Prerelease_EqualCoreVersion
    require_below_test.go:80: expected t.Fatal(), got none
--- FAIL: Test_RequireBelow_Prerelease_EqualCoreVersion (3.29s)
=== RUN   Test_RequireBelow_Prerelease_HigherCoreVersion
--- PASS: Test_RequireBelow_Prerelease_HigherCoreVersion (3.04s)
=== RUN   Test_RequireBelow_Prerelease_HigherPrerelease
--- PASS: Test_RequireBelow_Prerelease_HigherPrerelease (3.22s)
=== RUN   Test_RequireBelow_Prerelease_LowerCoreVersion
--- PASS: Test_RequireBelow_Prerelease_LowerCoreVersion (2.74s)
=== RUN   Test_RequireBelow_Prerelease_LowerPrerelease
--- PASS: Test_RequireBelow_Prerelease_LowerPrerelease (2.80s)

=== RUN   Test_RequireBetween
--- PASS: Test_RequireBetween (2.69s)
=== RUN   Test_RequireBetween_Error_BelowMin
--- PASS: Test_RequireBetween_Error_BelowMin (2.57s)
=== RUN   Test_RequireBetween_Error_EqToMax
--- PASS: Test_RequireBetween_Error_EqToMax (2.29s)
=== RUN   Test_RequireBetween_Prerelease_MaxEqualCoreVersion
    require_between_test.go:114: expected t.Fatal(), got none
--- FAIL: Test_RequireBetween_Prerelease_MaxEqualCoreVersion (3.24s)
=== RUN   Test_RequireBetween_Prerelease_MinEqualCoreVersion
    require_between_test.go:141: expected Terraform CLI version between 1.8.0 and 1.9.0 but detected version is 1.8.0-rc1
--- FAIL: Test_RequireBetween_Prerelease_MinEqualCoreVersion (2.83s)
=== RUN   Test_RequireBetween_Prerelease_MaxHigherCoreVersion
--- PASS: Test_RequireBetween_Prerelease_MaxHigherCoreVersion (3.17s)
=== RUN   Test_RequireBetween_Prerelease_MinHigherCoreVersion
--- PASS: Test_RequireBetween_Prerelease_MinHigherCoreVersion (3.09s)
=== RUN   Test_RequireBetween_Prerelease_MaxHigherPrerelease
--- PASS: Test_RequireBetween_Prerelease_MaxHigherPrerelease (3.84s)
=== RUN   Test_RequireBetween_Prerelease_MinHigherPrerelease
--- PASS: Test_RequireBetween_Prerelease_MinHigherPrerelease (3.02s)
=== RUN   Test_RequireBetween_Prerelease_MaxLowerCoreVersion
--- PASS: Test_RequireBetween_Prerelease_MaxLowerCoreVersion (2.84s)
=== RUN   Test_RequireBetween_Prerelease_MinLowerCoreVersion
--- PASS: Test_RequireBetween_Prerelease_MinLowerCoreVersion (4.07s)
=== RUN   Test_RequireBetween_Prerelease_MaxLowerPrerelease
--- PASS: Test_RequireBetween_Prerelease_MaxLowerPrerelease (2.79s)
=== RUN   Test_RequireBetween_Prerelease_MinLowerPrerelease
--- PASS: Test_RequireBetween_Prerelease_MinLowerPrerelease (3.25s)

=== RUN   Test_RequireNot
--- PASS: Test_RequireNot (3.70s)
=== RUN   Test_RequireNot_Error
--- PASS: Test_RequireNot_Error (2.76s)
=== RUN   Test_RequireNot_Prerelease_EqualCoreVersion
    require_not_test.go:73: expected t.Fatal(), got none
--- FAIL: Test_RequireNot_Prerelease_EqualCoreVersion (3.06s)
=== RUN   Test_RequireNot_Prerelease_HigherCoreVersion
--- PASS: Test_RequireNot_Prerelease_HigherCoreVersion (3.19s)
=== RUN   Test_RequireNot_Prerelease_HigherPrerelease
--- PASS: Test_RequireNot_Prerelease_HigherPrerelease (3.25s)
=== RUN   Test_RequireNot_Prerelease_LowerCoreVersion
--- PASS: Test_RequireNot_Prerelease_LowerCoreVersion (5.37s)
=== RUN   Test_RequireNot_Prerelease_LowerPrerelease
--- PASS: Test_RequireNot_Prerelease_LowerPrerelease (3.13s)

=== RUN   Test_SkipAbove_SkipTest
    skip_above_test.go:22: Terraform CLI version 1.3.0 is above maximum version 1.2.9: skipping test
--- SKIP: Test_SkipAbove_SkipTest (2.34s)
=== RUN   Test_SkipAbove_RunTest
--- PASS: Test_SkipAbove_RunTest (2.51s)
=== RUN   Test_SkipAbove_Prerelease_EqualCoreVersion
--- PASS: Test_SkipAbove_Prerelease_EqualCoreVersion (2.96s)
=== RUN   Test_SkipAbove_Prerelease_HigherCoreVersion
--- PASS: Test_SkipAbove_Prerelease_HigherCoreVersion (3.22s)
=== RUN   Test_SkipAbove_Prerelease_HigherPrerelease
--- PASS: Test_SkipAbove_Prerelease_HigherPrerelease (3.29s)
=== RUN   Test_SkipAbove_Prerelease_LowerCoreVersion
    skip_above_test.go:145: Terraform CLI version 1.8.0-rc1 is above maximum version 1.7.0: skipping test
--- SKIP: Test_SkipAbove_Prerelease_LowerCoreVersion (2.68s)
=== RUN   Test_SkipAbove_Prerelease_LowerPrerelease
    skip_above_test.go:166: Terraform CLI version 1.8.0-rc1 is above maximum version 1.8.0-beta1: skipping test
--- SKIP: Test_SkipAbove_Prerelease_LowerPrerelease (2.62s)

=== RUN   Test_SkipBetween_SkipTest
    skip_between_test.go:24: Terraform CLI version 1.2.0 is between 1.2.0 and 1.3.0: skipping test.
--- SKIP: Test_SkipBetween_SkipTest (2.70s)
=== RUN   Test_SkipBetween_RunTest_AboveMax
--- PASS: Test_SkipBetween_RunTest_AboveMax (2.71s)
=== RUN   Test_SkipBetween_RunTest_EqToMin
--- PASS: Test_SkipBetween_RunTest_EqToMin (2.76s)
=== RUN   Test_SkipBetween_Prerelease_MaxEqualCoreVersion
    skip_between_test.go:106: Terraform CLI version 1.8.0-rc1 is between 1.7.0 and 1.8.0: skipping test.
--- SKIP: Test_SkipBetween_Prerelease_MaxEqualCoreVersion (3.04s)
=== RUN   Test_SkipBetween_Prerelease_MinEqualCoreVersion
--- PASS: Test_SkipBetween_Prerelease_MinEqualCoreVersion (3.36s)
=== RUN   Test_SkipBetween_Prerelease_MaxHigherCoreVersion
    skip_between_test.go:154: Terraform CLI version 1.7.0-rc1 is between 1.6.0 and 1.8.0: skipping test.
--- SKIP: Test_SkipBetween_Prerelease_MaxHigherCoreVersion (2.82s)
=== RUN   Test_SkipBetween_Prerelease_MinHigherCoreVersion
--- PASS: Test_SkipBetween_Prerelease_MinHigherCoreVersion (3.81s)
=== RUN   Test_SkipBetween_Prerelease_MaxHigherPrerelease
    skip_between_test.go:198: Terraform CLI version 1.7.0-rc1 is between 1.6.0 and 1.7.0-rc2: skipping test.
--- SKIP: Test_SkipBetween_Prerelease_MaxHigherPrerelease (3.37s)
=== RUN   Test_SkipBetween_Prerelease_MinHigherPrerelease
--- PASS: Test_SkipBetween_Prerelease_MinHigherPrerelease (4.33s)
=== RUN   Test_SkipBetween_Prerelease_MaxLowerCoreVersion
--- PASS: Test_SkipBetween_Prerelease_MaxLowerCoreVersion (3.33s)
=== RUN   Test_SkipBetween_Prerelease_MinLowerCoreVersion
    skip_between_test.go:261: Terraform CLI version 1.8.0-rc1 is between 1.7.0 and 1.9.0: skipping test.
--- SKIP: Test_SkipBetween_Prerelease_MinLowerCoreVersion (3.12s)
=== RUN   Test_SkipBetween_Prerelease_MaxLowerPrerelease
--- PASS: Test_SkipBetween_Prerelease_MaxLowerPrerelease (3.08s)
=== RUN   Test_SkipBetween_Prerelease_MinLowerPrerelease
    skip_between_test.go:303: Terraform CLI version 1.8.0-rc1 is between 1.8.0-beta1 and 1.9.0: skipping test.
--- SKIP: Test_SkipBetween_Prerelease_MinLowerPrerelease (2.86s)

=== RUN   Test_SkipIf_SkipTest
    skip_if_test.go:24: Terraform CLI version is 1.4.3: skipping test.
--- SKIP: Test_SkipIf_SkipTest (2.52s)
=== RUN   Test_SkipIf_RunTest
--- PASS: Test_SkipIf_RunTest (3.15s)
=== RUN   Test_SkipIf_Prerelease_EqualCoreVersion
--- PASS: Test_SkipIf_Prerelease_EqualCoreVersion (4.58s)
=== RUN   Test_SkipIf_Prerelease_HigherCoreVersion
--- PASS: Test_SkipIf_Prerelease_HigherCoreVersion (3.46s)
=== RUN   Test_SkipIf_Prerelease_HigherPrerelease
--- PASS: Test_SkipIf_Prerelease_HigherPrerelease (3.28s)
=== RUN   Test_SkipIf_Prerelease_LowerCoreVersion
--- PASS: Test_SkipIf_Prerelease_LowerCoreVersion (3.10s)
=== RUN   Test_SkipIf_Prerelease_LowerPrerelease
--- PASS: Test_SkipIf_Prerelease_LowerPrerelease (3.06s)

Now (only showing skip testing as require testing will fail if incorrect):

=== RUN   Test_SkipAbove_SkipTest
    skip_above_test.go:22: Terraform CLI version 1.3.0 is above maximum version 1.2.9: skipping test
--- SKIP: Test_SkipAbove_SkipTest (2.87s)
=== RUN   Test_SkipAbove_RunTest
--- PASS: Test_SkipAbove_RunTest (2.66s)
=== RUN   Test_SkipAbove_Prerelease_EqualCoreVersion
--- PASS: Test_SkipAbove_Prerelease_EqualCoreVersion (2.98s)
=== RUN   Test_SkipAbove_Prerelease_HigherCoreVersion
--- PASS: Test_SkipAbove_Prerelease_HigherCoreVersion (3.19s)
=== RUN   Test_SkipAbove_Prerelease_HigherPrerelease
--- PASS: Test_SkipAbove_Prerelease_HigherPrerelease (3.29s)
=== RUN   Test_SkipAbove_Prerelease_LowerCoreVersion
    skip_above_test.go:145: Terraform CLI version 1.8.0-rc1 is above maximum version 1.7.0: skipping test
--- SKIP: Test_SkipAbove_Prerelease_LowerCoreVersion (3.32s)
=== RUN   Test_SkipAbove_Prerelease_LowerPrerelease
    skip_above_test.go:166: Terraform CLI version 1.8.0-rc1 is above maximum version 1.8.0-beta1: skipping test
--- SKIP: Test_SkipAbove_Prerelease_LowerPrerelease (2.72s)

=== RUN   Test_SkipBetween_SkipTest
    skip_between_test.go:24: Terraform CLI version 1.2.0 is between 1.2.0 and 1.3.0: skipping test.
--- SKIP: Test_SkipBetween_SkipTest (2.97s)
=== RUN   Test_SkipBetween_RunTest_AboveMax
--- PASS: Test_SkipBetween_RunTest_AboveMax (2.69s)
=== RUN   Test_SkipBetween_RunTest_EqToMin
--- PASS: Test_SkipBetween_RunTest_EqToMin (2.47s)
=== RUN   Test_SkipBetween_Prerelease_MaxEqualCoreVersion
--- PASS: Test_SkipBetween_Prerelease_MaxEqualCoreVersion (3.65s)
=== RUN   Test_SkipBetween_Prerelease_MinEqualCoreVersion
    skip_between_test.go:131: Terraform CLI version 1.8.0-rc1 is between 1.8.0 and 1.9.0: skipping test.
--- SKIP: Test_SkipBetween_Prerelease_MinEqualCoreVersion (3.10s)
=== RUN   Test_SkipBetween_Prerelease_MaxHigherCoreVersion
    skip_between_test.go:154: Terraform CLI version 1.7.0-rc1 is between 1.6.0 and 1.8.0: skipping test.
--- SKIP: Test_SkipBetween_Prerelease_MaxHigherCoreVersion (2.86s)
=== RUN   Test_SkipBetween_Prerelease_MinHigherCoreVersion
--- PASS: Test_SkipBetween_Prerelease_MinHigherCoreVersion (3.46s)
=== RUN   Test_SkipBetween_Prerelease_MaxHigherPrerelease
    skip_between_test.go:198: Terraform CLI version 1.7.0-rc1 is between 1.6.0 and 1.7.0-rc2: skipping test.
--- SKIP: Test_SkipBetween_Prerelease_MaxHigherPrerelease (3.04s)
=== RUN   Test_SkipBetween_Prerelease_MinHigherPrerelease
--- PASS: Test_SkipBetween_Prerelease_MinHigherPrerelease (3.39s)
=== RUN   Test_SkipBetween_Prerelease_MaxLowerCoreVersion
--- PASS: Test_SkipBetween_Prerelease_MaxLowerCoreVersion (3.01s)
=== RUN   Test_SkipBetween_Prerelease_MinLowerCoreVersion
    skip_between_test.go:261: Terraform CLI version 1.8.0-rc1 is between 1.7.0 and 1.9.0: skipping test.
--- SKIP: Test_SkipBetween_Prerelease_MinLowerCoreVersion (2.66s)
=== RUN   Test_SkipBetween_Prerelease_MaxLowerPrerelease
--- PASS: Test_SkipBetween_Prerelease_MaxLowerPrerelease (3.24s)
=== RUN   Test_SkipBetween_Prerelease_MinLowerPrerelease
    skip_between_test.go:303: Terraform CLI version 1.8.0-rc1 is between 1.8.0-beta1 and 1.9.0: skipping test.
--- SKIP: Test_SkipBetween_Prerelease_MinLowerPrerelease (2.91s)

=== RUN   Test_SkipIf_SkipTest
    skip_if_test.go:24: Terraform CLI version is 1.4.3: skipping test.
--- SKIP: Test_SkipIf_SkipTest (2.51s)
=== RUN   Test_SkipIf_RunTest
--- PASS: Test_SkipIf_RunTest (3.12s)
=== RUN   Test_SkipIf_Prerelease_EqualCoreVersion
    skip_if_test.go:70: Terraform CLI version is 1.8.0: skipping test.
--- SKIP: Test_SkipIf_Prerelease_EqualCoreVersion (2.51s)
=== RUN   Test_SkipIf_Prerelease_HigherCoreVersion
--- PASS: Test_SkipIf_Prerelease_HigherCoreVersion (3.22s)
=== RUN   Test_SkipIf_Prerelease_HigherPrerelease
--- PASS: Test_SkipIf_Prerelease_HigherPrerelease (3.27s)
=== RUN   Test_SkipIf_Prerelease_LowerCoreVersion
--- PASS: Test_SkipIf_Prerelease_LowerCoreVersion (3.03s)
=== RUN   Test_SkipIf_Prerelease_LowerPrerelease
--- PASS: Test_SkipIf_Prerelease_LowerPrerelease (3.15s)
github-actions[bot] commented 2 months ago

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active contributions. If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.