pulumi / pulumi-yaml

YAML language provider for Pulumi
Apache License 2.0
39 stars 12 forks source link

Quoted variable values incorrectly converted to integers #453

Closed jamest-gaia closed 11 months ago

jamest-gaia commented 1 year ago

What happened?

My Pulumi.develop.yaml file

config:
  aws:profile: 028111111111_AWSAdministratorAccess
  aws:region: ap-southeast-2
  myprojectname:aws_account_id: "028111111111" # I also tried single quotes here
  myprojectname:dns_domain: "redacted"
  myprojectname:aws_route53_zoneid: "redacted"

When I attempt to use this value aws_account_id in the code, it strips the zero. The stripping doesn't happen to the aws:profile. Perhaps because it contains some alpha characters too, or perhaps because the aws provider handles the variable differently.

Someone on Slack (Scott Lowe) suggested:

Do you have a reference to that configuration value in the config block of your Pulumi.yaml file? If so, you can specify a type property there (set it to “string”), and that may help resolve the issue.

I tried that, perhaps I'm missing something simple. So this valueaws_account_id is different for each stack hence its original location in the stack files i.e. Pulumi.develop.yaml, Pulumi.staging.yaml etc. I tried to implement your suggestion by adding a couple of lines under config: at the top of Pulumi.yaml like so: config:

  aws_account_id:
    type: string

Then I printed the output like so:

outputs:
  # Debug printouts
  accIDString: "${aws_account_id}"

And it's still getting converted.

Outputs:
  + accIDString               : 2.809102044e+10

I also tried a slightly different way, but this just printed the variable name

config:
  awsAccountID: 
    type: string
    default: ${aws_account_id}
...
outputs:
  # Debug printouts
  accIDString: "${awsAccountID}"
~~~~~~~~
Outputs:
  + accIDString               : "${aws_account_id}"

Expected Behavior

I expected variable values entered with quotes to stay as strings

Steps to reproduce

  1. Set a variable with an integer as the value, but cast it as a string.
  2. Try to use the value in the yaml code. It will get converted into a 'number'

This is where I tried to use it (in an IAM policy)

 # Allow to send emails with SES.
            - Action:
                - ses:SendEmail
              Effect: Allow
              Resource: "*" # "arn:aws:ses:${aws:region}:${aws_account_id}:identity/${ses_DomainIdentity.domain}" # can't use this due to bug with yaml stripping leading 0 from aws_account_id

Output of pulumi about

CLI          
Version      3.68.0 # (originally reported on 3.65.1)
Go Version   go1.20.4 # (originally reported on go1.20.3)
Go Compiler  gc

Plugins
NAME  VERSION
yaml  unknown

Host     
OS       debian
Version  11.5
Arch     x86_64

This project is written in yaml

Additional context

Have tried double and single quotes, have tried !!str '5432'

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

jamest-gaia commented 1 year ago

Another example of this happening. Variable definition (I've tried both double and single quotes): image

Variable usage: image

Have also tried:

- name: POSTGRES_PORT
  value: !!str "${confPOSTGRES_PORT}"

Error:

 aws:ecs:TaskDefinition (myFarGateService):
    error: aws:ecs/taskDefinition:TaskDefinition resource 'myFarGateService' has a problem: ECS Task Definition container_definitions is invalid: Error decoding JSON: json: cannot unmarshal number into Go struct field KeyValuePair.Environment.Value of type string. Examine values at 'TaskDefinition.ContainerDefinitions'.

Not sure if this is related? https://github.com/yaml/pyyaml/issues/98

Update: ended up hardcoding it.

image

jamest-gaia commented 1 year ago

Bump on this. It seems like it happens with booleans as well. Pulumi somehow converts things into a type. It's making me unable to deploy to ECS... Again, I've tried with double quotes, single quotes, and also trying to cast it using yaml !!str keyword Variable definition:

config:
  biosys:ALLOW_PUBLIC_REGISTRATION: !!str 'True'

Code:

environment:
            # https://www.pulumi.com/registry/packages/awsx/api-docs/ecs/fargatetaskdefinition/#taskdefinitionkeyvaluepair
            - name: ALLOW_PUBLIC_REGISTRATION
              value: !!str "${ALLOW_PUBLIC_REGISTRATION}"

Error:

aws:ecs:TaskDefinition (biosys-440-service):
    error: aws:ecs/taskDefinition:TaskDefinition resource 'biosys-440-service' has a problem: ECS Task Definition container_definitions is invalid: Error decoding JSON: json: cannot unmarshal bool into Go struct field KeyValuePair.Environment.Value of type string. Examine values at 'TaskDefinition.ContainerDefinitions'.
jamest-gaia commented 1 year ago

Seems like using this yaml syntax solves it

 biosys:ALLOW_PUBLIC_REGISTRATION: |-
   "True"

I found this out by using the pulumi config set command to add the variable to the stack file instead of adding it myself manually.

jamest-pin commented 11 months ago

This now seems to be happening in reverse. I have values I want to be booleans, but they're being force converted into strings.

jamest-pin commented 11 months ago

pulumi seems to be converting all stack config values to strings. I want to store a boolean type there. Is there a way to store a value as a boolean, or to convert a string to a bool? I tried forcing it but didn't work. I suppose it's possible that options: section doesn't accept variables..but I hope not.

# Pulumi.develop.yaml
config:
  myproject:myvar: false

# Pulumi.yaml

resources:
  myresource:
    options:
      protect: ${pulumi.stack}.${myvar}
    properties:
      ...

The above gives an error

error: failed to discover plugin requirements: 
-error: Pulumi.yaml:71,16-42: protect must be a boolean value;

If I try to force it:

 options:
      protect: !!bool ${pulumi.stack}.${myvar}

It doesn't work:

error: could not unmarshal '.../Pulumi.yaml': invalid YAML file: yaml: cannot decode !!str `${pulumi.stack}.${protect_dynamodb_tables}` as a !!bool

I also tried passing it through the variables: section in Pulumi.yaml

# Pulumi.yaml
...
variables:
  varProtectTablesAsBool:
    type: bool
    value: ${pulumi.stack}.${protect_dynamodb_tables} # tried both with !!bool and without

Still doesn't work, giving the first error protect must be a boolean value;

Frassle commented 11 months ago

I think protect is set to only accept a literal bool expression, not an interpolated expression. Some recent work in https://github.com/pulumi/pulumi-yaml/pull/510 to loosen this restriction for "version", probably doable for "protect" as well.