search

pulumi / pulumi-terraform-bridge

A library allowing Terraform providers to be bridged into Pulumi.
Apache License 2.0
194 stars 43 forks source link

Empty map incorrectly translates to null #1106

Open t0yv0 opened 1 year ago

t0yv0 commented 1 year ago

What happened?

The original issue https://github.com/pulumi/pulumi-aws/issues/2517#issuecomment-1544418955 uncovered a problem in the bridge. Under certain circumstances an empty object generated by the upstream provider turns into null on the wire when returned from the bridge, which is unexpected. A preferred form would use {} or {"field": null} possibly to represent this object.

Expected Behavior

Empty objects are represented in a way distinct from null.

Steps to reproduce

https://github.com/pulumi/pulumi-aws/issues/2517 has a full repro. We also have a unit test reproducing the issue available in https://github.com/pulumi/pulumi-terraform-bridge/pull/1105

Output of pulumi about

CLI          
Version      3.66.0
Go Version   go1.20.3
Go Compiler  gc

Plugins
NAME    VERSION
aws     5.40.0
python  unknown

Host     
OS       darwin
Version  13.1
Arch     x86_64

This project is written in python: executable='/usr/bin/python3' version='3.9.6
'

Current Stack: t0yv0/aws-2517/dev

TYPE                            URN
pulumi:pulumi:Stack             urn:pulumi:dev::aws-2517::pulumi:pulumi:Stack::aws-2517-dev
pulumi:providers:aws            urn:pulumi:dev::aws-2517::pulumi:providers:aws::default_5_40_0
aws:ec2/vpc:Vpc                 urn:pulumi:dev::aws-2517::aws:ec2/vpc:Vpc::main
aws:lb/targetGroup:TargetGroup  urn:pulumi:dev::aws-2517::aws:lb/targetGroup:TargetGroup::test
aws:lb/targetGroup:TargetGroup  urn:pulumi:dev::aws-2517::aws:lb/targetGroup:TargetGroup::foo

Found no pending operations associated with dev

Backend        
Name           pulumi.com
URL            https://app.pulumi.com/t0yv0
User           t0yv0
Organizations  t0yv0, pulumi

Dependencies:
NAME        VERSION
pip         23.1.2
pulumi-aws  5.40.0
setuptools  67.7.2
wheel       0.40.0

Pulumi locates its logs in /var/folders/gk/cchgxh512m72f_dmkcc3d09h0000gp/T/ by default

Additional context

This may be fixed by https://github.com/pulumi/pulumi-terraform-bridge/pull/887

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

t0yv0 commented 1 year ago

Some more context from debugging.

main.py:

import pulumi
import pulumi_aws as aws

main = aws.ec2.Vpc("main", cidr_block="10.0.0.0/16")
test = aws.lb.TargetGroup("test",
    port=80,
    protocol="HTTP",
    vpc_id=main.id)

pulumi.export('arn', test.arn)

repro.sh:

#!/usr/bin/env bash

set -euo pipefail

export AWS_PROFILE=devsandbox

pulumi destroy --yes
rm -rf *.json
pulumi up --yes --skip-preview

ARN=$(pulumi stack output arn)

# pulumi import aws:lb/targetGroup:TargetGroup foo "$ARN"

rm -rf out.txt

export PATH="/Users/t0yv0/code/pulumi-aws/bin:$PATH"

# TF_LOG=TRACE PULUMI_DEBUG_GRPC="$PWD/debug.json" pulumi import aws:lb/targetGroup:TargetGroup foo "$ARN" --yes --logtostderr --logflow -v=9 2> out.txt

PULUMI_DEBUG_GRPC="$PWD/debug.json" pulumi import aws:lb/targetGroup:TargetGroup foo "$ARN" --yes

Instrumented provider trace:

   Running v2Resource importer
    t aws_lb_target_group
    id arn:aws:elasticloadbalancing:us-east-1:616138583583:targetgroup/test-f9f76b1/b8fd8b6af0872ca2
    meta valast: cannot convert value of type func(*middleware.Stack) error
    meta type #*conns.AWSClient
    v2Results
    RESULT 0
    target_failover [] valast []interface{}{}
    runTerraformImporter returned this state
    map[id:arn:aws:elasticloadbalancing:us-east-1:616138583583:targetgroup/test-f9f76b1/b8fd8b6af0872ca2]
    map[string]interface{}{"id": "arn:aws:elasticloadbalancing:us-east-1:616138583583:targetgroup/test-f9f76b1/b8fd8b6af0872ca2"}
    Before RefreshWithoutUpgrade
    &InstanceState{
        ID: "arn:aws:elasticloadbalancing:us-east-1:616138583583:targetgroup/test-f9f76b1/b8fd8b6af0872ca2",
        Attributes: map[string]string{
                "health_check.#":    "0",
                "id":                "arn:aws:elasticloadbalancing:us-east-1:616138583583:targetgroup/test-f9f76b1/b8fd8b6af0872ca2",
                "stickiness.#":      "0",
                "target_failover.#": "0",
        },
        Meta: map[string]interface{}{"schema_version": "0"},
    }
    outputRaw
    {
      HealthCheckEnabled: true,
      HealthCheckIntervalSeconds: 30,
      HealthCheckPath: "/",
      HealthCheckPort: "traffic-port",
      HealthCheckProtocol: "HTTP",
      HealthCheckTimeoutSeconds: 5,
      HealthyThresholdCount: 5,
      IpAddressType: "ipv4",
      Matcher: {
        HttpCode: "200"
      },
      Port: 80,
      Protocol: "HTTP",
      ProtocolVersion: "HTTP1",
      TargetGroupArn: "arn:aws:elasticloadbalancing:us-east-1:616138583583:targetgroup/test-f9f76b1/b8fd8b6af0872ca2",
      TargetGroupName: "test-f9f76b1",
      TargetType: "instance",
      UnhealthyThresholdCount: 2,
      VpcId: "vpc-0af8bc8e0e370a7bd"
    }
    &TargetGroup{
        HealthCheckEnabled:         valast.Addr(true).(*bool),
        HealthCheckIntervalSeconds: valast.Addr(30).(*int64),
        HealthCheckPath:            valast.Addr("/").(*string),
        HealthCheckPort:            valast.Addr("traffic-port").(*string),
        HealthCheckProtocol:        valast.Addr("HTTP").(*string),
        HealthCheckTimeoutSeconds:  valast.Addr(5).(*int64),
        HealthyThresholdCount:      valast.Addr(5).(*int64),
        IpAddressType:              valast.Addr("ipv4").(*string),
        Matcher: &Matcher{
                HttpCode: valast.Addr("200").(*string),
        },
        Port:                    valast.Addr(80).(*int64),
        Protocol:                valast.Addr("HTTP").(*string),
        ProtocolVersion:         valast.Addr("HTTP1").(*string),
        TargetGroupArn:          valast.Addr("arn:aws:elasticloadbalancing:us-east-1:616138583583:targetgroup/test-f9f76b1/b8fd8b6af0872ca2").(*string),
        TargetGroupName:         valast.Addr("test-f9f76b1").(*string),
        TargetType:              valast.Addr("instance").(*string),
        UnhealthyThresholdCount: valast.Addr(2).(*int64),
        VpcId:                   valast.Addr("vpc-0af8bc8e0e370a7bd").(*string),
    }
    flattenTargetGroupFailover returns 1 element
    map[]
    map[string]interface{}{}
    After RefreshWithoutUpgrade
    &InstanceState{
        ID: "arn:aws:elasticloadbalancing:us-east-1:616138583583:targetgroup/test-f9f76b1/b8fd8b6af0872ca2",
        Attributes: map[string]string{
                "arn":                                "arn:aws:elasticloadbalancing:us-east-1:616138583583:targetgroup/test-f9f76b1/b8fd8b6af0872ca2",
                "arn_suffix":                         "targetgroup/test-f9f76b1/b8fd8b6af0872ca2",
                "deregistration_delay":               "300",
                "health_check.#":                     "1",
                "health_check.0.enabled":             "true",
                "health_check.0.healthy_threshold":   "5",
                "health_check.0.interval":            "30",
                "health_check.0.matcher":             "200",
                "health_check.0.path":                "/",
                "health_check.0.port":                "traffic-port",
                "health_check.0.protocol":            "HTTP",
                "health_check.0.timeout":             "5",
                "health_check.0.unhealthy_threshold": "2",
                "id":                                 "arn:aws:elasticloadbalancing:us-east-1:616138583583:targetgroup/test-f9f76b1/b8fd8b6af0872ca2",
                "ip_address_type":                    "ipv4",
                "load_balancing_algorithm_type":      "round_robin",
                "load_balancing_cross_zone_enabled":  "use_load_balancer_configuration",
                "name":                               "test-f9f76b1",
                "port":                               "80",
                "protocol":                           "HTTP",
                "protocol_version":                   "HTTP1",
                "slow_start":                         "0",
                "stickiness.#":                       "1",
                "stickiness.0.cookie_duration":       "86400",
                "stickiness.0.cookie_name":           "",
                "stickiness.0.enabled":               "false",
                "stickiness.0.type":                  "lb_cookie",
                "tags.%":                             "0",
                "tags_all.%":                         "0",
                "target_failover.#":                  "1",
                "target_type":                        "instance",
                "vpc_id":                             "vpc-0af8bc8e0e370a7bd",
        },
    }
    Refresh returned this state
    {0xc003622270 <nil>}
    v2InstanceState{tf: &InstanceState{
        ID: "arn:aws:elasticloadbalancing:us-east-1:616138583583:targetgroup/test-f9f76b1/b8fd8b6af0872ca2",
        Attributes: map[string]string{
                "arn":                                "arn:aws:elasticloadbalancing:us-east-1:616138583583:targetgroup/test-f9f76b1/b8fd8b6af0872ca2",
                "arn_suffix":                         "targetgroup/test-f9f76b1/b8fd8b6af0872ca2",
                "deregistration_delay":               "300",
                "health_check.#":                     "1",
                "health_check.0.enabled":             "true",
                "health_check.0.healthy_threshold":   "5",
                "health_check.0.interval":            "30",
                "health_check.0.matcher":             "200",
                "health_check.0.path":                "/",
                "health_check.0.port":                "traffic-port",
                "health_check.0.protocol":            "HTTP",
                "health_check.0.timeout":             "5",
                "health_check.0.unhealthy_threshold": "2",
                "id":                                 "arn:aws:elasticloadbalancing:us-east-1:616138583583:targetgroup/test-f9f76b1/b8fd8b6af0872ca2",
                "ip_address_type":                    "ipv4",
                "load_balancing_algorithm_type":      "round_robin",
                "load_balancing_cross_zone_enabled":  "use_load_balancer_configuration",
                "name":                               "test-f9f76b1",
                "port":                               "80",
                "protocol":                           "HTTP",
                "protocol_version":                   "HTTP1",
                "slow_start":                         "0",
                "stickiness.#":                       "1",
                "stickiness.0.cookie_duration":       "86400",
                "stickiness.0.cookie_name":           "",
                "stickiness.0.enabled":               "false",
                "stickiness.0.type":                  "lb_cookie",
                "tags.%":                             "0",
                "tags_all.%":                         "0",
                "target_failover.#":                  "1",
                "target_type":                        "instance",
                "vpc_id":                             "vpc-0af8bc8e0e370a7bd",
        },
    }}

Pulumi schema:

$ jq '.resources["aws:lb/targetGroup:TargetGroup"].properties["targetFailovers"]' "$S"
{
  "type": "array",
  "items": {
    "$ref": "#/types/aws:lb/TargetGroupTargetFailover:TargetGroupTargetFailover"
  },
  "description": "Target failover block. Only applicable for Gateway Load Balancer target groups. See target_failover for more information.\n"
}

jq '.types["aws:lb/TargetGroupTargetFailover:TargetGroupTargetFailover"]' "$S"
{
  "properties": {
    "onDeregistration": {
      "type": "string",
      "description": "Indicates how the GWLB handles existing flows when a target is deregistered. Possible values are `rebalance` and `no_rebalance`. Must match the attribute value set for `on_unhealthy`. Default: `no_rebalance`.\n"
    },
    "onUnhealthy": {
      "type": "string",
      "description": "Indicates how the GWLB handles existing flows when a target is unhealthy. Possible values are `rebalance` and `no_rebalance`. Must match the attribute value set for `on_deregistration`. Default: `no_rebalance`.\n"
    }
  },
  "type": "object",
  "required": [
    "onDeregistration",
    "onUnhealthy"
  ]
}

gRPC trace of the Read method.

{
  "method": "/pulumirpc.ResourceProvider/Read",
  "request": {
    "id": "arn:aws:elasticloadbalancing:us-east-1:616138583583:targetgroup/test-c530e0b/6aee61b47ab16785",
    "urn": "urn:pulumi:dev::aws-2517::aws:lb/targetGroup:TargetGroup::foo",
    "properties": {}
  },
  "response": {
    "id": "arn:aws:elasticloadbalancing:us-east-1:616138583583:targetgroup/test-c530e0b/6aee61b47ab16785",
    "properties": {
      "arn": "arn:aws:elasticloadbalancing:us-east-1:616138583583:targetgroup/test-c530e0b/6aee61b47ab16785",
      "arnSuffix": "targetgroup/test-c530e0b/6aee61b47ab16785",
      "deregistrationDelay": 300,
      "healthCheck": {
        "enabled": true,
        "healthyThreshold": 5,
        "interval": 30,
        "matcher": "200",
        "path": "/",
        "port": "traffic-port",
        "protocol": "HTTP",
        "timeout": 5,
        "unhealthyThreshold": 2
      },
      "id": "arn:aws:elasticloadbalancing:us-east-1:616138583583:targetgroup/test-c530e0b/6aee61b47ab16785",
      "ipAddressType": "ipv4",
      "loadBalancingAlgorithmType": "round_robin",
      "loadBalancingCrossZoneEnabled": "use_load_balancer_configuration",
      "name": "test-c530e0b",
      "port": 80,
      "protocol": "HTTP",
      "protocolVersion": "HTTP1",
      "slowStart": 0,
      "stickiness": {
        "cookieDuration": 86400,
        "cookieName": "",
        "enabled": false,
        "type": "lb_cookie"
      },
      "tags": {},
      "tagsAll": {},
      "targetFailovers": [
        null
      ],
      "targetType": "instance",
      "vpcId": "vpc-02f8ca754b4d39d01"
    },
    "inputs": {
      "__defaults": [],
      "deregistrationDelay": 300,
      "healthCheck": {
        "__defaults": [],
        "healthyThreshold": 5,
        "matcher": "200",
        "path": "/",
        "timeout": 5,
        "unhealthyThreshold": 2
      },
      "ipAddressType": "ipv4",
      "loadBalancingAlgorithmType": "round_robin",
      "loadBalancingCrossZoneEnabled": "use_load_balancer_configuration",
      "name": "test-c530e0b",
      "port": 80,
      "protocol": "HTTP",
      "protocolVersion": "HTTP1",
      "stickiness": {
        "__defaults": [],
        "enabled": false,
        "type": "lb_cookie"
      },
      "targetFailovers": [
        null
      ],
      "vpcId": "vpc-02f8ca754b4d39d01"
    }
  },
  "metadata": {
    "kind": "resource",
    "mode": "client",
    "name": "aws"
  }
}