pulumi / pulumi-aws

An Amazon Web Services (AWS) Pulumi resource package, providing multi-language access to AWS
Apache License 2.0
461 stars 155 forks source link

Type checking failure when using ec2.LaunchTemplate and the iamInstanceProfile #4360

Closed karlmutch-cynch closed 3 months ago

karlmutch-cynch commented 3 months ago

What happened?

When supplying an object to the function aws.ec2.LaunchTemplate(...) the iam_instance_profile parameter does not pass type checking.

It appears that what ever is passed into this parameter somehow becomes a string even if I supply a pulumi_aws.iam.instance_profile.InstanceProfile object

Example

    ecs_instance_profile = aws.iam.InstanceProfile(f"{cluster_name}_ecs_iam_instance_profile",
                                                   role=ecs_instance_role.name)

...

        gpu_launch_config = aws.ec2.LaunchTemplate(f"{cluster_name}_ec2_gpu_launch",
            instance_type=asg_gpu_class,
            image_id=aws.ec2.get_ami(most_recent=True, 
                                owners=["amazon"],
                                filters=[{
                                    "name":"name", 
                                    "values":["amzn2-ami-ecs-hvm-*-x86_64-ebs"]}]
                                ).id,  
            iam_instance_profile=ecs_instance_profile,
            vpc_security_group_ids=security_groups_ids,
            name_prefix=f"abacus-{cluster_name}-gpu-",
            user_data=f"""{user_data}
echo "ip_resolve=4" >> /etc/yum.conf
echo "ECS_ENABLE_GPU_SUPPORT=true" >>/etc/ecs/ecs.config 
            """)

...

$ pulumi up
     └─ aws:ec2:LaunchTemplate  devel_ec2_gpu_launch           1 error; 3 warnings

Diagnostics:
  aws:ec2:LaunchTemplate (devel_ec2_gpu_launch):
    warning: Type checking failed:
    warning: Unexpected type at field "iamInstanceProfile":
               expected object type, got string type
    warning: Type checking is still experimental. If you believe that a warning is incorrect,
    please let us know by creating an issue at https://github.com/pulumi/pulumi-terraform-bridge/issues.
    This will become a hard error in the future.
    error: aws:ec2/launchTemplate:LaunchTemplate resource 'devel_ec2_gpu_launch' has a problem: Expected Object Type. Expected object, got string. Examine values at 'devel_ec2_gpu_launch.iamInstanceProfile'.

Output of pulumi about

CLI Version 3.128.0 Go Version go1.22.5 Go Compiler gc

Plugins KIND NAME VERSION resource aws 6.41.0 resource docker 4.5.4 language python unknown

Host OS ubuntu Version 20.04 Arch x86_64

...

Backend Name pulumi.com URL https://app.pulumi.com/karlmutch-cynch User karlmutch-cynch Organizations karlmutch-cynch, cynch-ai Token type personal

Dependencies: NAME VERSION pip 24.1 pulumi_aws 6.41.0 pulumi_docker 4.5.4 setuptools 70.1.1 wheel 0.43.0

Additional context

No response

Contributing

Vote on this issue by adding a 👍 reaction. To contribute a fix for this issue, leave a comment (and link to your pull request, if you've opened one already).

iwahbe commented 3 months ago

Hi @karlmutch-cynch. Thanks for reporting an issue. We'll take a look soon.

corymhall commented 3 months ago

@karlmutch-cynch I think the error message might be backwards. Can you try specifying

iam_instance_profile=ecs_instance_profile.arn,
karlmutch-cynch commented 3 months ago

@corymhall It fails with the same error regardless of what I supply in that parameter.

karlmutch-cynch commented 3 months ago

Appears I need to do something like:

            iam_instance_profile={
                "arn":ecs_instance_profile.arn,
            },

Having said that the LaunchTemplate then yields other issues with the user_data parameter and base64 encoding not being right in there. Do you know of any working examples of using the new LaunchTemplate over the LaunchConfiguration ?

    error: 1 error occurred:
        * creating EC2 Launch Template (abacus-devel-gpu-20240809171606725300000001): operation error EC2: CreateLaunchTemplate, https response error StatusCode: 400, RequestID: c4bab31b-0fd2-4872-8426-ab286595d9ae, api error InvalidUserData.Malformed: Invalid BASE64 encoding of user data.
karlmutch-cynch commented 3 months ago

Also I am seeing when the LaunchTemplates are being applied by pulumi the default version being used by the auto scaling group is not incremented meaning the new launch template is never applied.

karlmutch-cynch commented 3 months ago

Also I am seeing when the LaunchTemplates are being applied by pulumi the default version being used by the auto scaling group is not incremented meaning the new launch template is never applied.

Determine that this has to be done using the update_default_version parameter to the LaunchTemplate constructor, its not the default. The original problem remains

guineveresaenger commented 3 months ago

Hi @karlmutch-cynch - thank you for your updates here. We're sorry that you ran into trouble with this. It does seem like the error message from the type check failure is correct, but it could absolutely be more helpful with regards to the expected shape of the resource field.

While it appears that our docs do not have a specific LaunchTemplate example, we do have a constructor example that may help here, and for future similar questions.

I have filed pulumi/pulumi-terraform-bridge#2307 to improve the error message in the future, especially so because you are correct, the pulumi-terraform bridge does transform your entry into a string under the hood, which is needlessly confusing when rendered as an error message.

In the meantime, I will move this issue to pulumi-aws, since any remaining issues pertain to that provider. It does sound like you have figured out the AWS - specific parts; if that is the case, feel free to close this issue.

flostadler commented 3 months ago

@karlmutch-cynch you managed to sort out the issue with iam_instance_profile, right?

To get the userdata working you'll need to base64 encode it.

In python you can do this like this:

import base64
encoded = base64.b64encode(b'your userdata string')
karlmutch-cynch commented 3 months ago

@karlmutch-cynch you managed to sort out the issue with iam_instance_profile, right?

To get the userdata working you'll need to base64 encode it.

In python you can do this like this:

import base64
encoded = base64.b64encode(b'your userdata string')

Yes that was sorted. Thanks