AWS IAM Role Creation Fails when Inline Policies are Included

phillipedwards commented 2 years ago

Hello!

Vote on this issue by adding a 👍 reaction
To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already)

Issue details

IAM Role creation fails when role definition includes inline-policies in the policies property.

Steps to reproduce

Versions Used:

Pulumi: v3.16.0
Pulumi AWS Native: 0.2.0

Create a new Pulumi AWS-Native Python project

Using below code:

  from pulumi_aws_native import iam
  import json

  role = iam.Role(
      "test-role",
      assume_role_policy_document=json.dumps(
      {
          "Version": "2012-10-17",
          "Statement": [
              {
                  "Action": "sts:AssumeRole",
                  "Effect": "Allow",
                  "Sid": "",
                  "Principal": {"Service": ["lambda.amazonaws.com"]},
              }
          ],
      }
  ),
      policies=[
          iam.RolePolicyArgs(
              policy_name="test-policy",
              policy_document=json.dumps(
                  {
                      "Version": "2012-10-17",
                      "Statement": [
                          {
                              "Action": ["ec2:Describe*"],
                              "Effect": "Allow",
                              "Resource": "*",
                          }
                      ],
                  }
              ),
          )
      ],
  )

Execute a pulumi up
See error output.

Expected: New IAM role with inline-policy is created and Pulumi program does not error. Actual: Role is created with inline-policy is created, but error occurs and role no longer becomes manageable from Pulumi.

Error Output

Furthermore, role does exist in AWS with inline-policy attached, but any attempt to delete role fails with:

emiliza commented 2 years ago

Linking to https://github.com/pulumi/pulumi-aws-native/issues/196 which is about the second half of the error here where delete roles fail for inline policies

mikhailshilkov commented 2 years ago

@phillipedwards Today I ran your code as-is and the deployment succeeded. I suspect this has been fixed on AWS Cloud Control side. I'll go ahead and close the issue - let me know if you still get problems.

jlgoolsbee commented 2 years ago

I can still reproduce this error with my own code and the example as-provided above. The resource is created successfully but it would seem the error is thrown via some post-creation routine; full stack trace below. I can also confirm the related issue (#196) is still present as well.

Pulumi version: v3.18.1 AWS Native provider version: v0.7.0

error: Traceback (most recent call last):
    File "/usr/local/bin/pulumi-language-python-exec", line 107, in <module>
    loop.run_until_complete(coro)
    File "/usr/local/Cellar/python@3.9/3.9.8/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
    return future.result()
    File "/Users/lgoolsbee/git/pulumi/.venv/lib/python3.9/site-packages/pulumi/runtime/stack.py", line 110, in run_in_stack
    await run_pulumi_func(lambda: Stack(func))
    File "/Users/lgoolsbee/git/pulumi/.venv/lib/python3.9/site-packages/pulumi/runtime/stack.py", line 45, in run_pulumi_func
    await wait_for_rpcs()
    File "/Users/lgoolsbee/git/pulumi/.venv/lib/python3.9/site-packages/pulumi/runtime/stack.py", line 94, in wait_for_rpcs
    raise exception
    File "/Users/lgoolsbee/git/pulumi/.venv/lib/python3.9/site-packages/pulumi/runtime/rpc_manager.py", line 65, in rpc_wrapper
    result = await rpc
    File "/Users/lgoolsbee/git/pulumi/.venv/lib/python3.9/site-packages/pulumi/output.py", line 94, in is_value_known
    return await is_known and not contains_unknowns(await future)
    File "/Users/lgoolsbee/git/pulumi/.venv/lib/python3.9/site-packages/pulumi/output.py", line 94, in is_value_known
    return await is_known and not contains_unknowns(await future)
    File "/Users/lgoolsbee/git/pulumi/.venv/lib/python3.9/site-packages/pulumi/output.py", line 94, in is_value_known
    return await is_known and not contains_unknowns(await future)
    [Previous line repeated 8 more times]
    File "/Users/lgoolsbee/git/pulumi/.venv/lib/python3.9/site-packages/pulumi/runtime/resource.py", line 568, in do_register
    rpc.resolve_outputs(res, resolver.serialized_props, resp.object, deps, resolvers, transform_using_type_metadata)
    File "/Users/lgoolsbee/git/pulumi/.venv/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 850, in resolve_outputs
    translated_value = translate_output_properties(value, translate_to_pass, types.get(key),
    File "/Users/lgoolsbee/git/pulumi/.venv/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 749, in translate_output_properties
    return [
    File "/Users/lgoolsbee/git/pulumi/.venv/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 750, in <listcomp>
    translate_output_properties(v,
    File "/Users/lgoolsbee/git/pulumi/.venv/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 712, in translate_output_properties
    translated_values = {
    File "/Users/lgoolsbee/git/pulumi/.venv/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 713, in <dictcomp>
    k: translate_output_properties(v,
    File "/Users/lgoolsbee/git/pulumi/.venv/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 733, in translate_output_properties
    raise AssertionError((f"Unexpected type; expected a value of type `{typ}`"
AssertionError: Unexpected type; expected a value of type `<class 'str'>` but got a value of type `<class 'dict'>` at resource `test-role`, property `policies.0.policyDocument`: {'version': '2012-10-17', 'statement': [{'resource': '*', 'effect': 'Allow', 'action': ['ec2:Describe*']}]}
error: an unhandled error occurred: Program exited with non-zero exit code: 1

wofl commented 2 years ago

Same happening for me:

error: Traceback (most recent call last):
      File "/pulumi/bin/pulumi-language-python-exec", line 107, in <module>
        loop.run_until_complete(coro)
      File "/usr/local/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
        return future.result()
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/stack.py", line 126, in run_in_stack
        await run_pulumi_func(lambda: Stack(func))
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/stack.py", line 51, in run_pulumi_func
        await wait_for_rpcs()
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/stack.py", line 73, in wait_for_rpcs
        await RPC_MANAGER.rpcs.pop()
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/rpc_manager.py", line 68, in rpc_wrapper
        result = await rpc
      File "/usr/local/lib/python3.9/site-packages/pulumi/output.py", line 98, in is_value_known
        return await is_known and not contains_unknowns(await future)
      File "/usr/local/lib/python3.9/site-packages/pulumi/output.py", line 98, in is_value_known
        return await is_known and not contains_unknowns(await future)
      File "/usr/local/lib/python3.9/site-packages/pulumi/output.py", line 98, in is_value_known
        return await is_known and not contains_unknowns(await future)
      [Previous line repeated 25 more times]
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/resource.py", line 514, in do_register
        resolver = await prepare_resource(res, ty, custom, remote, props, opts, typ)
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/resource.py", line 124, in prepare_resource
        serialized_props = await rpc.serialize_properties(
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 172, in serialize_properties
        result = await serialize_property(
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 248, in serialize_property
        "urn": await serialize_property(
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 340, in serialize_property
        is_known = await output._is_known
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/rpc_manager.py", line 68, in rpc_wrapper
        result = await rpc
      File "/usr/local/lib/python3.9/site-packages/pulumi/output.py", line 98, in is_value_known
        return await is_known and not contains_unknowns(await future)
      File "/usr/local/lib/python3.9/site-packages/pulumi/output.py", line 98, in is_value_known
        return await is_known and not contains_unknowns(await future)
      File "/usr/local/lib/python3.9/site-packages/pulumi/output.py", line 98, in is_value_known
        return await is_known and not contains_unknowns(await future)
      [Previous line repeated 17 more times]
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/resource.py", line 514, in do_register
        resolver = await prepare_resource(res, ty, custom, remote, props, opts, typ)
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/resource.py", line 124, in prepare_resource
        serialized_props = await rpc.serialize_properties(
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 172, in serialize_properties
        result = await serialize_property(
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 340, in serialize_property
        is_known = await output._is_known
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/rpc_manager.py", line 68, in rpc_wrapper
        result = await rpc
      File "/usr/local/lib/python3.9/site-packages/pulumi/output.py", line 98, in is_value_known
        return await is_known and not contains_unknowns(await future)
      File "/usr/local/lib/python3.9/site-packages/pulumi/output.py", line 98, in is_value_known
        return await is_known and not contains_unknowns(await future)
      File "/usr/local/lib/python3.9/site-packages/pulumi/output.py", line 98, in is_value_known
        return await is_known and not contains_unknowns(await future)
      [Previous line repeated 8 more times]
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/resource.py", line 653, in do_register
        rpc.resolve_outputs(
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 981, in resolve_outputs
        translated_value = translate_output_properties(
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 863, in translate_output_properties
        return [
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 864, in <listcomp>
        translate_output_properties(
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 819, in translate_output_properties
        translated_values = {
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 820, in <dictcomp>
        k: translate_output_properties(
      File "/usr/local/lib/python3.9/site-packages/pulumi/runtime/rpc.py", line 842, in translate_output_properties
        raise AssertionError(
    AssertionError: Unexpected type; expected a value of type `<class 'str'>` but got a value of type `<class 'dict'>` at resource `ecs-task-execution`, property `policies.0.policyDocument`: {'version': '201
2-10-17', 'statement': [{'effect': 'Allow', 'resource': '*', 'action': 'logs:CreateLogGroup'}]}
    error: an unhandled error occurred: Program exited with non-zero exit code: 1

SpecLad commented 1 year ago

Here's what I think is going on here.

The PolicyDocument property is defined in the schema as follows:

https://github.com/pulumi/pulumi-aws-native/blob/360bc11ff2538e17bacfb34c512cd1b34ef7ba50/aws-cloudformation-schema/aws-iam-role.json#L12-L15

So the Cloud Control API allows either an object, or a JSON-encoded string to be supplied when creating the Role resource. But (I tested this), when you query the resource back, PolicyDocument is always returned as an object.

However, it seems that the AWS Native generator ignores all listed types after the first one, so the generated code thinks that PolicyDocument must be a string. This can be seen here:

https://github.com/pulumi/pulumi-aws-native/blob/360bc11ff2538e17bacfb34c512cd1b34ef7ba50/sdk/python/pulumi_aws_native/iam/_inputs.py#L93

Consequently, when Pulumi creates the resource, it works (because the API accepts the string), but when it queries it back, it gets an object instead of a string, so type validation fails.

The ideal fix for this would be to make the generator able to process alternative types. But failing that, it would at least be nice if it could treat [ "string", "object" ] the same way it treats [ "object", "string" ] - that is, as "object".

pulumi / pulumi-aws-native