Optional properties with ConflictsWith and defaults expected to be null

[danielrbradley]

In the Azure DevOps provider, we've encountered an issue where the upstream provider explicitly expects values to be null.

The schema for the sub-property is defined as:

            vgVariable: {
                Type:     schema.TypeSet,
                Required: true,
                MinItems: 1,
                Elem: &schema.Resource{
                    Schema: map[string]*schema.Schema{
                        vgName: {
                            Type:     schema.TypeString,
                            Required: true,
                        },
                        vgValue: {
                            Type:          schema.TypeString,
                            Optional:      true,
                            Default:       "",
                            ConflictsWith: []string{vgKeyVault},
                        },
                        secretVgValue: {
                            Type:          schema.TypeString,
                            Optional:      true,
                            Sensitive:     true,
                            Default:       "",
                            ConflictsWith: []string{vgKeyVault},
                        },
                        vgIsSecret: {
                            Type:          schema.TypeBool,
                            Optional:      true,
                            Default:       false,
                            ConflictsWith: []string{vgKeyVault},
                        },
                        vgContentType: {
                            Type:     schema.TypeString,
                            Computed: true,
                        },
                        vgEnabled: {
                            Type:     schema.TypeBool,
                            Computed: true,
                        },
                        vgExpires: {
                            Type:     schema.TypeString,
                            Computed: true,
                        },
                    },
                },
            },

Sample program:

name: variable-group
runtime: yaml

resources:
  exampleInsights:
    type: azuredevops:index/variableGroup:VariableGroup
    properties:
      projectId: af8eabf7-65c9-4ccd-afb7-a4e136af29e8
      name: variablegroupname
      description: my description
      allowAccess: false
      variables:
        - name: key1
          value: val1

During the create and update the following code is executed which appears to work correctly in TF but fails via the bridge:

    variables := d.Get(vgVariable).(*schema.Set).List()

    // needed to detect if the secret_value attribute is set in the config
    // see https://github.com/hashicorp/terraform-plugin-sdk/issues/741
    for it := d.GetRawConfig().AsValueMap()[vgVariable].ElementIterator(); it.Next(); {
        _, ctyVariable := it.Element()
        ctyVariableAsMap := ctyVariable.AsValueMap()
        name := ctyVariableAsMap[vgName].AsString()
        valueSet := !ctyVariableAsMap[vgValue].IsNull()
        secretValueSet := !ctyVariableAsMap[secretVgValue].IsNull()
        isSecretSet := !ctyVariableAsMap[vgIsSecret].IsNull()

        if valueSet && (secretValueSet || isSecretSet) || secretValueSet != isSecretSet {
            return nil, nil, fmt.Errorf("`%s` variable can have either only `value` attribute or both `is_secret` and `secret_value` attributes", name)
        }
    }

The variables inside the loop resolve to:

gRPC request:

{
  "method": "/pulumirpc.ResourceProvider/Create",
  "request": {
    "urn": "urn:pulumi:test::variable-group::azuredevops:index/variableGroup:VariableGroup::exampleInsights",
    "properties": {
      "__defaults": [],
      "allowAccess": false,
      "description": "my description",
      "name": "variablegroupname",
      "projectId": "af8eabf7-65c9-4ccd-afb7-a4e136af29e8",
      "variables": [
        {
          "__defaults": [
            "isSecret",
            "secretValue"
          ],
          "isSecret": false,
          "name": "key1",
          "secretValue": "",
          "value": "val1"
        }
      ]
    },
    "name": "exampleInsights",
    "type": "azuredevops:index/variableGroup:VariableGroup"
  },
  "errors": [
    "rpc error: code = Unknown desc = 1 error occurred:\n\t* Expanding variable group resource data: `key1` variable can have either only `value` attribute or both `is_secret` and `secret_value` attributes\n\n"
  ],
  "metadata": {
    "kind": "resource",
    "mode": "client",
    "name": "azuredevops"
  }
}

Props in the Pulumi provider ready to be sent to the TF provider

resource.PropertyMap [
  "__defaults": {V: interface {}([]github.com/pulumi/pulumi/sdk/v3/go/common/resource.PropertyValue) *(*interface {})(0x140011663e8)}, 
  "allowAccess": {V: interface {}(bool) *(*interface {})(0x140011663f8)}, 
  "description": {V: interface {}(string) *(*interface {})(0x14001166408)}, 
  "name": {V: interface {}(string) *(*interface {})(0x14001166418)}, 
  "projectId": {V: interface {}(string) *(*interface {})(0x14001166428)}, 
  "variables": {V: interface {}([]github.com/pulumi/pulumi/sdk/v3/go/common/resource.PropertyValue) 
    [(*"github.com/pulumi/pulumi/sdk/v3/go/common/resource.PropertyValue")(resource.PropertyValue {V: interface {}(github.com/pulumi/pulumi/sdk/v3/go/common/resource.PropertyMap) [
      "__defaults": (*"github.com/pulumi/pulumi/sdk/v3/go/common/resource.PropertyValue")([]resource.PropertyValue len: 2, cap: 2, [{V: interface {}(string) *(*interface {})(0x1400081c4a0)},{V: interface {}(string) *(*interface {})(0x1400081c4b0)}]),
      "isSecret": (*"github.com/pulumi/pulumi/sdk/v3/go/common/resource.PropertyValue")(interface {}(bool) false),
      "name": (*"github.com/pulumi/pulumi/sdk/v3/go/common/resource.PropertyValue")(interface {}(string) "key1"),
      "secretValue": (*"github.com/pulumi/pulumi/sdk/v3/go/common/resource.PropertyValue")(interface {}(string) ""),
      "value": (*"github.com/pulumi/pulumi/sdk/v3/go/common/resource.PropertyValue")(interface {}(string) "val1"), ]}
    )]}, ]

Provider issue:

• https://github.com/pulumi/pulumi-azuredevops/issues/514

[VenelinMartinov]

Thanks for the detailed issue!

[markdebeer]

Is there a rough ETA on when this fix will be implemented?

[VenelinMartinov]

@markdebeer we do not have an ETA. Which provider are you using which hits this problem? If this is in AzureDevOps, can you please comment on the issue there with your use case and we might be able to prioritize a workaround instead.

EDIT: my bad, just noticed you opened the issue https://github.com/pulumi/pulumi-azuredevops/issues/514

[VenelinMartinov]

@danielrbradley can we consider a workaround for the problem in the provider?