Feature request: Support LanguageExtensions feature Fn::ForEach

[twasink]

It would be useful if the LanguageExtensions transform feature Fn::ForEach was supported with SAM CLI.

The documentation for the LanguageExtensions transforms states that you can use it with the Serverless transform:

If you're using multiple transforms, use a list format. If you're using custom macros, place AWS-provided transforms after your custom macros. If you're using both the AWS::LanguageExtensions and AWS::Serverless transforms, the AWS::LanguageExtensions transform must come before the AWS::Serverless transform in the list.

However, when you try to use the example for the Fn::ForEach function (with or without include the Serverless Transform), the sam-cli fails to understand the syntax.

Example template.yaml:

AWSTemplateFormatVersion: 2010-09-09
Transform: 
  - - AWS::LanguageExtensions
    - AWS::Serverless-2016-10-31
Resources:
  'Fn::ForEach::Topics':
    - TopicName
    - - Success
      - Failure
      - Timeout
      - Unknown
    - 'SnsTopic${TopicName}':
        Type: 'AWS::SNS::Topic'
        Properties:
          TopicName: !Ref TopicName
          FifoTopic: true

Example output from sam build:

$ sam build
2023-07-30 17:52:37 Plugin 'ServerlessAppPlugin' raised an exception: 'list' object has no attribute 'get'
Traceback (most recent call last):
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samtranslator/plugins/sam_plugins.py", line 130, in act
    getattr(plugin, method_name)(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samtranslator/metrics/method_decorator.py", line 116, in wrapper_cw_timer
    exec_result = func(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samtranslator/plugins/application/serverless_app_plugin.py", line 124, in on_before_transform_template
    for logical_id, app in template.iterate({SamResourceType.Application.value}):
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samtranslator/sdk/template.py", line 34, in iterate
    resource = SamResource(resource_dict)
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samtranslator/sdk/resource.py", line 26, in __init__
    self.type = resource_dict.get("Type")
AttributeError: 'list' object has no attribute 'get'

Error: 'list' object has no attribute 'get'
Traceback:
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/click/decorators.py", line 84, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samcli/lib/telemetry/metric.py", line 184, in wrapped
    raise exception  # pylint: disable=raising-bad-type
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samcli/lib/telemetry/metric.py", line 149, in wrapped
    return_value = func(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samcli/lib/utils/version_checker.py", line 42, in wrapped
    actual_result = func(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samcli/cli/main.py", line 95, in wrapper
    return func(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samcli/commands/build/command.py", line 166, in cli
    do_cli(
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samcli/commands/build/command.py", line 238, in do_cli
    with BuildContext(
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samcli/commands/build/build_context.py", line 184, in __enter__
    self.set_up()
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samcli/commands/build/build_context.py", line 190, in set_up
    self._stacks, remote_stack_full_paths = SamLocalStackProvider.get_stacks(
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samcli/lib/providers/sam_stack_provider.py", line 269, in get_stacks
    current = SamLocalStackProvider(
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samcli/lib/providers/sam_stack_provider.py", line 61, in __init__
    self._template_dict = self.get_template(
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samcli/lib/providers/sam_base_provider.py", line 193, in get_template
    template_dict = SamTranslatorWrapper(template_dict, parameter_values=parameters_values).run_plugins()
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samcli/lib/samlib/wrapper.py", line 73, in run_plugins
    parser.parse(template_copy, all_plugins)  # parse() will run all configured plugins
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samcli/lib/samlib/wrapper.py", line 130, in parse
    sam_plugins.act(LifeCycleEvents.before_transform_template, sam_template)
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samtranslator/plugins/sam_plugins.py", line 136, in act
    raise ex
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samtranslator/plugins/sam_plugins.py", line 130, in act
    getattr(plugin, method_name)(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samtranslator/metrics/method_decorator.py", line 116, in wrapper_cw_timer
    exec_result = func(*args, **kwargs)
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samtranslator/plugins/application/serverless_app_plugin.py", line 124, in on_before_transform_template
    for logical_id, app in template.iterate({SamResourceType.Application.value}):
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samtranslator/sdk/template.py", line 34, in iterate
    resource = SamResource(resource_dict)
  File "/usr/local/Cellar/aws-sam-cli/1.94.0/libexec/lib/python3.8/site-packages/samtranslator/sdk/resource.py", line 26, in __init__
    self.type = resource_dict.get("Type")

An unexpected error was encountered while executing "sam build".
Search for an existing issue:
https://github.com/aws/aws-sam-cli/issues?q=is%3Aissue+is%3Aopen+Bug%3A%20sam%20build%20-%20AttributeError
Or create a bug report:
https://github.com/aws/aws-sam-cli/issues/new?template=Bug_report.md&title=Bug%3A%20sam%20build%20-%20AttributeError

(This was with SAM CLI version 1.94)

[moelasmar]

Thanks @twasink for raising this issue. I will discuss this issue with the team to see how can we prioritize it.

[cortexcompiler]

I am also running into this. Would be nice to use Fn::ForEach to do some things similar to SAM serverless resources. I was hoping to add a group of AWS::Serverless::Function resources along with essentially templated LogGroup SubscriptionFilter resources for each of the functions. However, I immediately ran into this error trying to run sam build

[cortexcompiler]

Also, is this really a feature request? I would expect SAM to support native CloudFormation functions. I feel like this is actually a bug.

[AEkman]

I can confirm that I have seen the same issue. When SAM is validating I get the following error: Error: 'list' object has no attribute 'get'

[AEkman]

It would also be good to mention that it's currently not supported in documentation!

[ggiallo28]

I am running the same issue with "aws cloudformation package"

[maciejGolebio]

[ryanniuu]

same issue when running "sam validate" i would say this is a bug not feature request

[nelsonmora48]

Ya vamos por la 1.98 y aun no se corrige este problema

[francisreyes-tfs]

Pull request 8096 from aws-cli looks promising but needs review

[rhbecker]

PR 8096 (aws-cli) was merged in mid-December. A couple of comments on related issues here and here indicate that recent aws-cli versions no longer have this problem.

Is the aws-cli "upstream" relative to sam-cli, meaning that we'll inevitably see this same fix in an imminent sam-cli releaes? Or does the sam-cli team need to independently address this issue?

Regardless, I'm hoping this might be near the finish line, so that I don't need to train our staff on 2 different stack management tools. 🤞

[mlmland]

I'm having this issue as well. Any updates?

[michal-sa]

PR 8096 (aws-cli) was merged in mid-December. A couple of comments on related issues here and here indicate that recent aws-cli versions no longer have this problem.

Is the aws-cli "upstream" relative to sam-cli, meaning that we'll inevitably see this same fix in an imminent sam-cli releaes? Or does the sam-cli team need to independently address this issue?

Regardless, I'm hoping this might be near the finish line, so that I don't need to train our staff on 2 different stack management tools. 🤞

I'm still seeing this issue with:

• aws --version: aws-cli/2.15.15 Python/3.11.6 Linux/5.14.0-1055-oem exe/x86_64.ubuntu.20 prompt/off
• sam --version: SAM CLI, version 1.108.0

Feels like, even with the fixed aws-cli, it needs to be fixed for sam-cli as well.

[alexandrosgkesos]

Still doesn't work with $ sam --version SAM CLI, version 1.111.0

[rhbecker]

@moelasmar Please consider changing this issue's label from type/feature to type/bug.

The language used on SAM's main marketing page, under the Why SAM? header, reinforces what several participants have asserted - i.e. that this issue is a bug ...

You can also define resources using CloudFormation in your SAM template and use the full suite of resources, intrinsic functions, and other template features that are available in AWS CloudFormation.

In reviewing this repo's issue history, I observe that type/feature issues are rarely resolved, while type/bug issues receive a good amount of attention.

[rapala61]

Hi everyone, I'm just here to bump this bug to keep it in the radar. I have a ~1000 line template that could be reduced to ~100 if this issue was resolved.

[scolebrook]

Bump!

https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/sam-specification-intrinsic-functions.html

The SAM docs state that adding the AWS::LanguageExtensions transform resolves it's lack of support for some intrinsic functions.

Intrinsic functions are built-in functions that enable you to assign values to properties that are only available at runtime. AWS SAM has limited support for certain intrinsic function properties, so it is unable to resolve some intrinsic functions. Consequently, we recommend adding the AWS::LanguageExtensions transform to resolve this:

This issue is either a bug in the documentation (it should clearly state what intrinsic functions aren't supported even when using AWS::LanguageExtensions) or it's a bug in SAM not functioning as documented.

CloudFormation itself can correctly resolve the transforms with both Language Extensions and Serverless listed in that order.

[prower-turnitin]

Please work on this. ForEach seems pretty basic function to me. If it can't be done with AWS::LanguageExtensions can SAM have these features built into itself?

[wasabideveloper]

+1

Hi, I am running into the same issue here. Please change the issue type to type/bug and give this issue more attention. This is definitely a bug, because the documentation clearly indicates that SAM supports all CloudFormation features. Thanks!

Details regarding my case if it helps: Template:

AWSTemplateFormatVersion: "2010-09-09"
Transform:
  - AWS::LanguageExtensions
  - AWS::Serverless-2016-10-31 # Removing Serverless transformation does not change anything

# ...

Resources:
  ReadOnlyGroup:
    Type: AWS::IdentityStore::Group
    Properties:
      Description: Read-only (all accounts)
      DisplayName: read-only
      IdentityStoreId: !Ref identityStoreId

  "Fn::ForEach::ReadOnlyGroupAssignment":
    - AccountId
    - !Split [",", !Ref allAccountIds]
    - "ReadOnlyGroupAssignment${AccountId}":
        Type: AWS::SSO::Assignment
        Properties:
          InstanceArn: !Ref identityCenterInstanceArn
          PermissionSetArn: !GetAtt ReadOnlyPermissionSet.PermissionSetArn
          TargetId: !Ref AccountId
          TargetType: "AWS_ACCOUNT"
          PrincipalType: "GROUP"
          PrincipalId: !GetAtt ReadOnlyGroup.GroupId

Console output of sam build:

Starting Build use cache                                                                                                                   
2024-07-05 08:13:52 Plugin 'ServerlessAppPlugin' raised an exception: 'list' object has no attribute 'get'
Traceback (most recent call last):
  File "samtranslator/plugins/sam_plugins.py", line 130, in act
  File "samtranslator/metrics/method_decorator.py", line 116, in wrapper_cw_timer
  File "samtranslator/plugins/application/serverless_app_plugin.py", line 124, in on_before_transform_template
  File "samtranslator/sdk/template.py", line 34, in iterate
  File "samtranslator/sdk/resource.py", line 26, in __init__
AttributeError: 'list' object has no attribute 'get'

Error: 'list' object has no attribute 'get'
Traceback:
  File "click/core.py", line 1055, in main
  File "click/core.py", line 1657, in invoke
  File "click/core.py", line 1404, in invoke
  File "click/core.py", line 760, in invoke
  File "click/decorators.py", line 84, in new_func
  File "click/core.py", line 760, in invoke
  File "samcli/lib/telemetry/metric.py", line 184, in wrapped
  File "samcli/lib/telemetry/metric.py", line 149, in wrapped
  File "samcli/lib/utils/version_checker.py", line 42, in wrapped
  File "samcli/cli/main.py", line 95, in wrapper
  File "samcli/commands/build/command.py", line 166, in cli
  File "samcli/commands/build/command.py", line 238, in do_cli
  File "samcli/commands/build/build_context.py", line 184, in __enter__
  File "samcli/commands/build/build_context.py", line 190, in set_up
  File "samcli/lib/providers/sam_stack_provider.py", line 280, in get_stacks
  File "samcli/lib/providers/sam_stack_provider.py", line 269, in get_stacks
  File "samcli/lib/providers/sam_stack_provider.py", line 61, in __init__
  File "samcli/lib/providers/sam_base_provider.py", line 193, in get_template
  File "samcli/lib/samlib/wrapper.py", line 73, in run_plugins
  File "samcli/lib/samlib/wrapper.py", line 130, in parse
  File "samtranslator/plugins/sam_plugins.py", line 136, in act
  File "samtranslator/plugins/sam_plugins.py", line 130, in act
  File "samtranslator/metrics/method_decorator.py", line 116, in wrapper_cw_timer
  File "samtranslator/plugins/application/serverless_app_plugin.py", line 124, in on_before_transform_template
  File "samtranslator/sdk/template.py", line 34, in iterate
  File "samtranslator/sdk/resource.py", line 26, in __init__

An unexpected error was encountered while executing "sam build".
Search for an existing issue:
https://github.com/aws/aws-sam-cli/issues?q=is%3Aissue+is%3Aopen+Bug%3A%20sam%20build%20-%20AttributeError
Or create a bug report:
https://github.com/aws/aws-sam-cli/issues/new?template=Bug_report.md&title=Bug%3A%20sam%20build%20-%20AttributeError