[BUG] Stack trace from the terraform-provider-megaport_v1.1.1 plugin: panic: runtime error

[gfysaris]

Describe the bug

$ terraform --version
Terraform v1.9.5
on linux_amd64
$ terraform init -upgrade
Initializing the backend...
Initializing provider plugins...
- Finding megaport/megaport versions matching "1.1.1"...
- Finding latest version of hashicorp/external...
- Installing megaport/megaport v1.1.1...
- Installed megaport/megaport v1.1.1 (self-signed, key ID C6F2DC9F50135458)
- Installing hashicorp/external v2.3.3...
- Installed hashicorp/external v2.3.3 (signed by HashiCorp)
Partner and community providers are signed by their developers.
If you'd like to know more about provider signing, you can read about it here:
https://www.terraform.io/docs/cli/plugins/signing.html
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
$ terraform validate
Success! The configuration is valid.
$ terraform plan
data.external.megaport_partners["OUTSCALE"]: Reading...
data.external.megaport_partners["ORACLE"]: Reading...
data.external.get_all_megaport_locations: Reading...
data.external.megaport_partners["AWS"]: Reading...
data.external.megaport_partners["AWSHC"]: Reading...
data.external.megaport_partners["AZURE"]: Reading...
data.external.megaport_partners["GOOGLE"]: Reading...
data.external.megaport_partners["IBM"]: Reading...
data.megaport_location.dc["Global Switch London East"]: Reading...
data.megaport_location.dc["Equinix LD5"]: Reading...
data.external.megaport_partners["OUTSCALE"]: Read complete after 2s [id=-]
data.external.megaport_partners["IBM"]: Read complete after 2s [id=-]
data.external.megaport_partners["AWS"]: Read complete after 2s [id=-]
data.external.megaport_partners["ORACLE"]: Read complete after 3s [id=-]
data.external.megaport_partners["GOOGLE"]: Read complete after 3s [id=-]
data.external.megaport_partners["AWSHC"]: Read complete after 3s [id=-]
data.external.megaport_partners["AZURE"]: Read complete after 3s [id=-]
data.megaport_partner.partner["AZURE | Amsterdam Secondary | e9d1b8ad-31a0-4d7c-95e0-956b8[55](https://gitlab.com/closebrothers/iac/megaport/cbl/-/jobs/7674204001#L55)011f2"]: Reading...
data.megaport_partner.partner["AWS | US East (N. Virginia) (us-east-1) | 0722ba2a-6dbb-486e-9416-f212abd8d94f"]: Reading...
data.megaport_partner.partner["AWSHC | Asia Pacific (Sydney) (ap-southeast-2) | e5f28d51-c8c9-46fa-951a-ac3ce81[59](https://gitlab.com/closebrothers/iac/megaport/cbl/-/jobs/7674204001#L59)061"]: Reading...
.....
.....
.....
data.megaport_partner.partner["AWS | Asia Pacific (Sydney) (ap-southeast-2) | 12c2499f-07cb-44ba-80c9-669097de8867"]: Read complete after 1s
data.megaport_partner.partner["AWSHC | EU (London) (eu-west-2) | e29e4a21-c317-48d8-8428-9110658a0bcb"]: Reading...
data.megaport_partner.partner["AWS | US East (N. Virginia) (us-east-1) | b2b7d8c6-5bf8-4260-8925-f8a66e2f4a86"]: Read complete after 1s
data.megaport_partner.partner["GOOGLE | Montreal (yul-zone2-1944) | a39cc950-a90e-44f0-8307-015c5a4a9266"]: Reading...
megaport_port.port["CBL-DC1-1"]: Preparing import... [id=796e3b0a-1eea-4c46-9e46-8e5c5bc5ce8c]
megaport_port.port["CBL-DC2-1"]: Preparing import... [id=f8d2a6ea-6075-4253-bf7c-d6cc08fac810]
Planning failed. Terraform encountered an error while generating this plan.
╷
│ Error: Request cancelled
│ 
│ The plugin6.(*GRPCProvider).ImportResourceState request was cancelled.
╵
╷
│ Error: Request cancelled
│ 
│ The plugin6.(*GRPCProvider).ImportResourceState request was cancelled.
╵
Stack trace from the terraform-provider-megaport_v1.1.1 plugin:
panic: runtime error: index out of range [0] with length 0
goroutine 1175 [running]:
github.com/megaport/terraform-provider-megaport/internal/provider.(*partnerPortDataSource).Read(0xc000064030, {0xdc9de8, 0xc000b0f920}, {{{{0xdce5a8, 0xc00098f440}, {0xb79ec0, 0xc00098f170}}, {0xdd0a20, 0xc00056d180}}, {{{0x0, ...}, ...}, ...}}, ...)
    github.com/megaport/terraform-provider-megaport/internal/provider/partner_port_data_source.go:281 +0x1a05
github.com/hashicorp/terraform-plugin-framework/internal/fwserver.(*Server).ReadDataSource(0xc00032a000, {0xdc9de8, 0xc000b0f920}, 0xc000b0ff80, 0xc0009b96c8)
    github.com/hashicorp/terraform-plugin-framework@v1.7.0/internal/fwserver/server_readdatasource.go:79 +0x433
github.com/hashicorp/terraform-plugin-framework/internal/proto6server.(*Server).ReadDataSource(0xc00032a000, {0xdc9de8?, 0xc000b0f620?}, 0xc000a20060)
    github.com/hashicorp/terraform-plugin-framework@v1.7.0/internal/proto6server/server_readdatasource.go:55 +0x41c
github.com/hashicorp/terraform-plugin-go/tfprotov6/tf6server.(*server).ReadDataSource(0xc0002325a0, {0xdc9de8?, 0xc000887650?}, 0xc000c98230)
    github.com/hashicorp/terraform-plugin-go@v0.22.1/tfprotov6/tf6server/server.go:686 +0x417
github.com/hashicorp/terraform-plugin-go/tfprotov6/internal/tfplugin6._Provider_ReadDataSource_Handler({0xc516e0?, 0xc0002325a0}, {0xdc9de8, 0xc000887650}, 0xc000c00000, 0x0)
    github.com/hashicorp/terraform-plugin-go@v0.22.1/tfprotov6/internal/tfplugin6/tfplugin6_grpc.pb.go:572 +0x169
google.golang.org/grpc.(*Server).processUnaryRPC(0xc0001a7000, {0xdc9de8, 0xc0008874a0}, {0xdcef80, 0xc00032aea0}, 0xc000af25a0, 0xc000321740, 0x12c6540, 0x0)
    google.golang.org/grpc@v1.62.1/server.go:1386 +0xe23
google.golang.org/grpc.(*Server).handleStream(0xc0001a7000, {0xdcef80, 0xc00032aea0}, 0xc000af25a0)
    google.golang.org/grpc@v1.62.1/server.go:1797 +0x100c
google.golang.org/grpc.(*Server).serveStreams.func2.1()
    google.golang.org/grpc@v1.62.1/server.go:1027 +0x8b
created by google.golang.org/grpc.(*Server).serveStreams.func2 in goroutine 28
    google.golang.org/grpc@v1.62.1/server.go:1038 +0x135
Error: The terraform-provider-megaport_v1.1.1 plugin crashed!
This is always indicative of a bug within the plugin. It would be immensely
helpful if you could report the crash with the plugin's maintainers so that it
can be fixed. The output above should help diagnose the issue.

Terraform and Provider Version

Terraform v1.9.5 on linux_amd64

Initializing provider plugins...
- Installed megaport/megaport v1.1.1 (self-signed, key ID C6F2DC9F50135458)
- Installed hashicorp/external v2.3.3 (signed by HashiCorp)

Terraform File and Steps to Reproduce

data.external.megaport_partners.tf

locals {
  megaport_partner_connect_types = [
    "IBM",
    "AWS",
    "AWSHC",
    "AZURE",
    "GOOGLE",
    "ORACLE",
    "OUTSCALE"
  ]
}

data "external" "megaport_partners" {
  for_each = toset(local.megaport_partner_connect_types)

  program = ["bash", "${path.module}/get_all_megaport_partners.sh"]

  query = {
    mgprt_api_access_key = var.MGPRT_API_ACCESS_KEY
    mgprt_api_key_secret = var.MGPRT_API_KEY_SECRET
    mgprt_cloud_provider = each.key
  }
}

locals {
  megaport_partners_grouped = (
    {
      for partner_connect_type in keys(data.external.megaport_partners)
      : "${partner_connect_type}" => jsondecode(base64decode(data.external.megaport_partners["${partner_connect_type}"].result.encoded_megaport_partners))
    }
  )

  megaport_partners = (
    merge(
      [
        for partner_connect_type in keys(local.megaport_partners_grouped)
        : {
          for partner in local.megaport_partners_grouped[partner_connect_type]
          : "${partner_connect_type} | ${partner.product_name} | ${partner.product_uid}" => partner
        }
      ]...
    )
  )

}

get_all_megaport_partners.sh

#!/bin/bash

## Exit if any of the intermediate steps fail
# set -e
set -euo pipefail

## Extract "foo" and "baz" arguments from the input into
## FOO and BAZ shell variables.
## jq will ensure that the values are properly quoted
## and escaped for consumption by the shell.
# eval "$(jq -r '@sh "FOO=\(.foo) BAZ=\(.baz)"')"
eval "$(jq -r '@sh "MGPRT_API_ACCESS_KEY=\(.mgprt_api_access_key) MGPRT_API_KEY_SECRET=\(.mgprt_api_key_secret) MGPRT_CLOUD_PROVIDER=\(.mgprt_cloud_provider)"')"

## Placeholder for whatever data-fetching logic your script implements
# FOOBAZ="$FOO $BAZ"
generate_megaport_api_token=$(curl --silent --request POST \
    --header "Content-Type: application/x-www-form-urlencoded" \
    --user "$MGPRT_API_ACCESS_KEY:$MGPRT_API_KEY_SECRET" \
    --data-urlencode "grant_type=client_credentials" \
    "https://auth-m2m.megaport.com/oauth2/token")

encoded_generate_megaport_api_token=$(echo $generate_megaport_api_token | base64 -w 0)

megaport_api_token=$(echo $generate_megaport_api_token | jq -r '.access_token')
encoded_megaport_api_token=$(echo $megaport_api_token | base64 -w 0)

get_all_megaport_partners=$(curl --silent --request GET \
    --header "Content-Type: application/json" \
    --header "Authorization: Bearer $megaport_api_token" \
    "https://api.megaport.com/v2/dropdowns/partner/megaports?connectType=$MGPRT_CLOUD_PROVIDER")

# Instead of passing the entire JSON as an argument to jq
# Write the JSON to a temporary file
megaport_partners=$(mktemp)
echo "$get_all_megaport_partners" > "$megaport_partners"

# Use jq with input redirection instead of passing the JSON as an argument
# filtered_megaport_partners_details=$(jq '[.data[] | .name]' "$megaport_partners")
filtered_megaport_partners_details=$(jq '[.data[] | {product_uid: .productUid, product_name: .title, company_uid: .companyUid, company_name: .companyName, location_id: .locationId, connect_type: .connectType}]' "$megaport_partners")

# Remove the temporary file
rm "$megaport_partners"

# Output the result
# all_megaport_partners=$(echo "{\"megaport_partners\": $filtered_megaport_partners_details}")

encoded_get_all_megaport_partners=$(echo $filtered_megaport_partners_details | base64 -w 0)

## Safely produce a JSON object containing the result value.
## jq will ensure that the value is properly quoted
## and escaped to produce a valid JSON string.
# jq -n --arg foobaz "$FOOBAZ" '{"foobaz":$foobaz}'

jq -n \
  --arg encd_generate_megaport_api_token "$encoded_generate_megaport_api_token" \
  --arg encd_megaport_partners "$encoded_get_all_megaport_partners" \
  --arg encd_megaport_api_token "$encoded_megaport_api_token" \
  '{
    encoded_megaport_api_token: $encd_megaport_api_token,
    encoded_megaport_partners: $encd_megaport_partners
  }'

data.megaport_partner.tf

data "megaport_partner" "partner" {
  for_each = local.megaport_partners

  connect_type = each.value.connect_type
  company_uid  = each.value.company_uid
  company_name = each.value.company_name
  product_name = each.value.product_name
  location_id  = each.value.location_id
}

Additional context Based on troubleshoot analysis from Gitlab AI

Root cause of failure The root cause of this job failure appears to be a crash in the Terraform provider plugin for Megaport (terraform-provider-megaport_v1.1.1). Specifically, the error occurs during the execution of the ReadDataSource function in the partnerPortDataSource struct.

The crash is caused by a panic due to an "index out of range" error, which suggests that the code is trying to access an element in a slice or array that doesn't exist. The exact line causing the issue is in the file partner_port_data_source.go at line 281.

The error message indicates:

panic: runtime error: index out of range [0] with length 0

This means that the code is trying to access the first element (index 0) of a slice or array that is empty (length 0).

The crash in the provider plugin causes Terraform to fail, as it cannot complete the operation it was trying to perform (likely reading data from a data source).

Example Fix To fix this issue, the Megaport Terraform provider code needs to be updated to handle the case where the slice or array might be empty. Here's an example of how the fix might look:

// File: partner_port_data_source.go
// Around line 281

func (d *partnerPortDataSource) Read(ctx context.Context, req datasource.ReadRequest, resp *datasource.ReadResponse) {
    // ... (previous code)

    // Assuming 'someSlice' is the slice causing the issue
    if len(someSlice) > 0 {
        // Perform the operation with someSlice[0]
        result := someSlice[0]
        // Use 'result' as needed
    } else {
        // Handle the case where the slice is empty
        resp.Diagnostics.AddError(
            "No data available",
            "The required data is not available. Please check your configuration.",
        )
        return
    }

    // ... (rest of the function)
}

This example adds a check to ensure that the slice has at least one element before trying to access it. If the slice is empty, it adds an error diagnostic and returns from the function.

To implement this fix, the Megaport Terraform provider maintainers would need to:

Identify the exact slice or array causing the issue. Add appropriate checks to handle cases where the data might be empty. Implement error handling or default values as appropriate for their use case. Test the changes thoroughly to ensure they resolve the issue without introducing new problems. Release a new version of the provider with the fix. Users of the Terraform Megaport provider would then need to update to the new version once it's released to resolve this issue in their CI/CD pipelines.

[mega-alex]

Appreciate the detailed error report, thank you! I believe this was already fixed with #135, but it wasn't yet included in a release. I've tagged and released v1.1.2 which should address this, would you mind giving that a test to make sure it resolves your issue?

[gfysaris]

Thanks for the update Alex! Indeed i've just tested with 1.1.3 and it worked. Thank you