error working with terraform framework custom schema type implementing `basetypes.ListValuable`

ocobles commented 4 months ago

What happened?

Error message: error: Failed to prepare a resource encoder: cannot derive an encoder for resource "equinix_metal_connection": Cannot derive an encoder for property "ports": issue deriving an object encoder: Cannot derive an encoder for property "virtual_circuit_ids": when deriving converters for an element of a collection: LookupSchemaPath failed at walk.NewSchemaPath(): walk.ElementStep{} is not applicable
schema definition of the attribute failing: https://github.com/equinix/pulumi-equinix/blob/a9d8f324dfa6fabbbb92d017d4ef8dcf677ef3bb/provider/cmd/pulumi-resource-equinix/schema.json#L7350-L7355
Terraform schema: https://github.com/equinix/terraform-provider-equinix/blob/1ef2417122f8c92b09d8d36c10e3e8473322f11b/internal/resources/metal/connection/models.go#L73
linked issue https://github.com/equinix/pulumi-equinix/issues/94

Example

-

Output of `pulumi about`

-

Additional context

This is a special case, in the Equinix terraform provider we are migrating from sdkv2 to framework, using tf5muxserver. Due to some limitations with protocol 5, we define the ports field with CustomType, which is a list of objects of type PortModel. As the object is not a schema, the type of the subfields is not defined and therefore types.List cannot be used for the virtual_circuit_ids subfield and it needs a typed list, that is why we use a custom list of strings that implements ListValuable

I have been able to verify that the AWS provider also fails in resources that use the same approach, for example

https://github.com/hashicorp/terraform-provider-aws/blob/eae23a82dc115e5abddba32f9de79a8899909f38/internal/service/bedrockagent/agent.go#L124

https://github.com/hashicorp/terraform-provider-aws/blob/eae23a82dc115e5abddba32f9de79a8899909f38/internal/service/bedrockagent/agent.go#L564

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

iwahbe commented 4 months ago

Hi @ocobles. Thanks for reporting this issue. We'll take a look. It definitely looks like a bridge bug.

iwahbe commented 4 months ago

I was able to reproduce the issue against AWS with the program you gave:

import pulumi
import pulumi_aws as aws

aws.bedrock.AgentAgent("agentAgentResource",
    agent_name="string",
    agent_resource_role_arn="string",
    foundation_model="string",
    customer_encryption_key_arn="string",
    description="string",
    idle_session_ttl_in_seconds=0,
    instruction="string",
    prepare_agent=False,
    prompt_override_configurations=[
        aws.bedrock.AgentAgentPromptOverrideConfigurationArgs(
            override_lambda="string",
            prompt_configurations=["any"],
        )
    ],
    tags={
        "string": "string",
    },
    timeouts=aws.bedrock.AgentAgentTimeoutsArgs(
        create="string",
        delete="string",
        update="string",
    ),
)

Diagnostics:
  aws:bedrock:AgentAgent (agentAgentResource):
    error: Failed to prepare a resource encoder: cannot derive an encoder for resource "aws_bedrockagent_agent": Cannot derive an encoder for property "prompt_override_configuration": issue deriving an object encoder: Cannot derive an encoder for property "prompt_configurations": when deriving converters for an element of a collection: LookupSchemaPath failed at walk.NewSchemaPath(): walk.ElementStep{} is not applicable

flostadler commented 3 months ago

This is also happening for aws:medialive/getInputDestination:getInputDestination in the AWS provider, causing https://github.com/pulumi/pulumi-aws/issues/4091.

Schema defintion, the property that's failing is vpcs:

"aws:medialive/getInputDestination:getInputDestination": {
            "properties": {
                "ip": {
                    "type": "string"
                },
                "port": {
                    "type": "string"
                },
                "url": {
                    "type": "string"
                },
                "vpcs": {
                    "type": "array",
                    "items": {
                        "$ref": "pulumi.json#/Any"
                    }
                }
            },
            "type": "object",
            "required": [
                "ip",
                "port",
                "url",
                "vpcs"
            ],
            "language": {
                "nodejs": {
                    "requiredInputs": []
                }
            }
        },

Here's the terraform schema: https://github.com/hashicorp/terraform-provider-aws/blob/4c52b0c1a9abf53acadf23f30c35f8b81215ef4b/internal/service/medialive/input_data_source.go#L132

Repro:

import * as aws from "@pulumi/aws";

const name = 'example';

const inputSecurityGroup = new aws.medialive.InputSecurityGroup(name, {
    whitelistRules: [
        {
            cidr: '1.1.1.1/32',
        },
    ],
});

const input = new aws.medialive.Input(name, {
    name,
    type: 'RTMP_PUSH',
    destinations: [
        {
            streamName: `live/${name}`,
        },
        {
            streamName: `live/${name}-backup`,
        },
    ],
    inputSecurityGroups: [inputSecurityGroup.id],
});

export const medialiveInputArn = aws.medialive.getInputOutput({
    id: input.id,
}).apply(d => d.arn);

Error:

error: Running program '/Users/flo/development/medialive-error/index.ts' failed with an unhandled exception:
    <ref *1> Error: invocation of aws:medialive/getInput:getInput returned an error: cannot derive an encoder for data source "aws_medialive_input": Cannot derive an encoder for property "destinations": issue deriving an object encoder: Cannot derive an encoder for property "vpc": when deriving converters for an element of a collection: LookupSchemaPath failed at walk.NewSchemaPath(): walk.ElementStep{} is not applicable
        at Object.callback (/Users/flo/development/medialive-error/node_modules/@pulumi/runtime/invoke.ts:172:37)
        at Object.onReceiveStatus (/Users/flo/development/medialive-error/node_modules/@grpc/grpc-js/src/client.ts:360:26)
        at Object.onReceiveStatus (/Users/flo/development/medialive-error/node_modules/@grpc/grpc-js/src/client-interceptors.ts:458:34)
        at Object.onReceiveStatus (/Users/flo/development/medialive-error/node_modules/@grpc/grpc-js/src/client-interceptors.ts:419:48)
        at /Users/flo/development/medialive-error/node_modules/@grpc/grpc-js/src/resolving-call.ts:163:24
        at processTicksAndRejections (node:internal/process/task_queues:77:11) {
      promise: Promise { <rejected> [Circular *1] }
    }

iwahbe commented 3 months ago

Thanks for letting us know @flostadler. We've (@guineveresaenger) has been looking at this and we believe the error is in schemashim.typeSchema.Elem:

https://github.com/pulumi/pulumi-terraform-bridge/blob/4cecc872578b930e1e12bb72eaa4332f6c05401d/pf/internal/schemashim/type_schema.go#L59-L67

It should be checking for basetypes.ListTypable instead of types.List. We should have a PR in the next couple of days.

pulumi-bot commented 2 months ago

This issue has been addressed in PR #2142 and shipped in release v3.86.0.

pulumi / pulumi-terraform-bridge