(*GRPCProvider).ValidateProviderConfig call at v3.5.0

I upgraded to v3.5.0 and got an error.

Terraform Version

users $ tv
Terraform v1.7.0
on linux_amd64
+ provider registry.terraform.io/pagerduty/pagerduty v3.5.0

Terraform Configuration Files

my config & resources are

terraform {
  required_providers {
    pagerduty = {
      source = "PagerDuty/pagerduty"
    }
  }
}

// and others

Panic Output

Planning failed. Terraform encountered an error while generating this plan.

╷
│ Error: Plugin did not respond
│
│   with provider["registry.terraform.io/pagerduty/pagerduty"],
│   on versions.tf line 9, in provider "pagerduty":
│    9: provider "pagerduty" {
│
│ The plugin encountered an error, and failed to respond to the plugin.(*GRPCProvider).ValidateProviderConfig call. The plugin logs may contain more details.
╵

Stack trace from the terraform-provider-pagerduty_v3.5.0 plugin:

panic: can't convert (schema.SchemaDefaultFunc)(0xb02200) to cty.Value

goroutine 51 [running]:
github.com/hashicorp/terraform-plugin-sdk/v2/internal/configs/hcl2shim.HCL2ValueFromConfigValue({0xee21a0?, 0xc0003d3680?})
        github.com/hashicorp/terraform-plugin-sdk/v2@v2.31.0/internal/configs/hcl2shim/values.go:231 +0x3ae
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).PrepareProviderConfig.func1({0xc000498350, 0xc0002482ab?, 0x0?}, {{{0x12a57c0?, 0xc0002482ab?}}, {0x0?, 0x0?}})
        github.com/hashicorp/terraform-plugin-sdk/v2@v2.31.0/helper/schema/grpc_provider.go:219 +0x157
github.com/hashicorp/go-cty/cty.transform({0xc000498350?, 0x1, 0x1}, {{{0x12a57c0?, 0xc0002482ab?}}, {0x0?, 0x0?}}, 0xc00060f5b8)
        github.com/hashicorp/go-cty@v1.4.1-0.20200414143053-d3edf31b6320/cty/walk.go:181 +0x8fb
github.com/hashicorp/go-cty/cty.transform({0x0?, 0x0, 0x0}, {{{0x12a58a0?, 0xc0004a6030?}}, {0xee9820?, 0xc00048e390?}}, 0xc0004475b8)
        github.com/hashicorp/go-cty@v1.4.1-0.20200414143053-d3edf31b6320/cty/walk.go:168 +0xaa5
github.com/hashicorp/go-cty/cty.Transform(...)
        github.com/hashicorp/go-cty@v1.4.1-0.20200414143053-d3edf31b6320/cty/walk.go:89
github.com/hashicorp/terraform-plugin-sdk/v2/helper/schema.(*GRPCProviderServer).PrepareProviderConfig(0xc0003d6660, {0x12a4f00?, 0xc0006328d0?}, 0xc000636020)
        github.com/hashicorp/terraform-plugin-sdk/v2@v2.31.0/helper/schema/grpc_provider.go:178 +0x1fb
github.com/hashicorp/terraform-plugin-go/tfprotov5/tf5server.(*server).PrepareProviderConfig(0xc0004260a0, {0x12a4f00?, 0xc0006a6480?}, 0xc0006a64b0)
        github.com/hashicorp/terraform-plugin-go@v0.20.0/tfprotov5/tf5server/server.go:568 +0x29c
github.com/hashicorp/terraform-plugin-go/tfprotov5/internal/tfplugin5._Provider_PrepareProviderConfig_Handler({0xff7d40?, 0xc0004260a0}, {0x12a4f00, 0xc0006a6480}, 0xc0006ce000, 0x0)
        github.com/hashicorp/terraform-plugin-go@v0.20.0/tfprotov5/internal/tfplugin5/tfplugin5_grpc.pb.go:377 +0x170
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0001b5000, {0x12a4f00, 0xc0006a63f0}, {0x12a96f8, 0xc0004829c0}, 0xc0006b8240, 0xc0003f3c50, 0x1934010, 0x0)                                                                                                                                google.golang.org/grpc@v1.60.1/server.go:1372 +0xe49
google.golang.org/grpc.(*Server).handleStream(0xc0001b5000, {0x12a96f8, 0xc0004829c0}, 0xc0006b8240)                                                                                                                                                                                            google.golang.org/grpc@v1.60.1/server.go:1783 +0x1031
google.golang.org/grpc.(*Server).serveStreams.func2.1()
        google.golang.org/grpc@v1.60.1/server.go:1016 +0x68
created by google.golang.org/grpc.(*Server).serveStreams.func2
        google.golang.org/grpc@v1.60.1/server.go:1027 +0x12e

Error: The terraform-provider-pagerduty_v3.5.0 plugin crashed!

This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

Expected Behavior

plan success

Actual Behavior

terraform init -upgrade is panic

Important Factoids

maybe no good this?

https://github.com/PagerDuty/terraform-provider-pagerduty/compare/v3.4.1...v3.5.0#diff-e89d2a475e625c3ef9a0dcd81045258920993498cb6a22eb3cb116ad18e3de09R42

It passed by adding the following code

users $ git diff versions.tf
diff --git a/terraform/pagerduty/users/versions.tf b/terraform/pagerduty/users/versions.tf
index a16e94d..deaab80 100644
--- a/terraform/pagerduty/users/versions.tf
+++ b/terraform/pagerduty/users/versions.tf
@@ -5,3 +5,8 @@ terraform {
     }
   }
 }
+
+provider "pagerduty" {
+  service_region = ""
+}
+

Until now, we have not written provider blocks because the defaults were fine.

The above process requires a provider block, and also requires the addition of service_region and the passing of an empty string. The service_region should not be required.

PagerDuty / terraform-provider-pagerduty