ModuleDefaultVersion and ResourceDefaultVersion always fail to List resources with CloudControl API

[benbridts]

Example of the failure:

~$  aws cloudcontrol list-resources --type-name AWS::CloudFormation::ModuleDefaultVersion

An error occurred (GeneralServiceException) when calling the ListResources operation:
AWS::CloudFormation::ModuleDefaultVersion Handler returned status FAILED: The security token included in the request is
invalid (Service: CloudFormation, Status Code: 403, Request ID: cb58282f-0c05-4079-8927-21e93a1db58a, Extended
Request ID: null) (HandlerErrorCode: GeneralServiceException, RequestToken: 6cfc2c86-69b9-453a-b813-8ada1f276d81)

~$ aws cloudformation list-types --visibility PRIVATE --type MODULE
{
    "TypeSummaries": [
        {
            "Type": "MODULE",
            "TypeName": "Sample::Generic::StaticWebsite::MODULE",
            "DefaultVersionId": "00000001",
            "TypeArn": "arn:aws:cloudformation:eu-west-1:123456789012:type/module/Sample-Generic-StaticWebsite-MODULE",
            "LastUpdated": "2020-11-24T20:31:48.196000+00:00",
            "Description": "Schema for Module Fragment of type Sample::Generic::StaticWebsite::MODULE"
        }
    ]
}

[PatMyron]

Over 100 resource types failed to list resources through cloudcontrol when I tested with:

import boto3
import json
import requests
client = boto3.client('cloudcontrol', region_name='us-east-1')
for resource_type in sorted(requests.get('https://d1uauaxba7bl26.cloudfront.net/latest/gzip/CloudFormationResourceSpecification.json').json()['ResourceTypes'].keys()):
    try:
        client.list_resources(
            TypeName=resource_type,
            MaxResults=100,
        )['ResourceDescriptions']
    except client.exceptions.UnsupportedActionException:
        pass
    except Exception as e:
        print(e)

including more CloudFormation resource types:

An error occurred (GeneralServiceException) when calling the ListResources operation: AWS::CloudFormation::ModuleDefaultVersion Handler returned status FAILED: The security token included in the request is invalid (Service: CloudFormation, Status Code: 403, Request ID: 8cc18188-6682-4bc6-a20a-0a45bad3ce75, Extended Request ID: null) (HandlerErrorCode: GeneralServiceException, RequestToken: 9b6960a0-8de9-488a-885e-0a763b2832cc)
An error occurred (ResourceNotFoundException) when calling the ListResources operation: AWS::CloudFormation::Publisher Handler returned status FAILED: The account [616281649822] is not registered as a publisher, if you want to describe a specific publisher, please specify the publisherID (Service: CloudFormation, Status Code: 400, Request ID: d5848bd0-009e-4613-a666-feb5948a9eec, Extended Request ID: null) (HandlerErrorCode: NotFound, RequestToken: c167bdfd-72cb-4849-bcf3-89eee06f6d66)
An error occurred (AccessDeniedException) when calling the ListResources operation: AWS::CloudFormation::ResourceDefaultVersion Handler returned status FAILED: The security token included in the request is invalid (Service: CloudFormation, Status Code: 403, Request ID: ff44921b-0839-422d-a723-656c8936b431, Extended Request ID: null) (HandlerErrorCode: AccessDenied, RequestToken: 92149b0e-f2e6-441f-8551-1e677ddc6b64)
An error occurred (InvalidRequestException) when calling the ListResources operation: Missing Or Invalid ResourceModel property in AWS::CloudFormation::ResourceVersion list handler request input. Required property: [TypeArn] or [TypeName]
An error occurred (GeneralServiceException) when calling the ListResources operation: AWS::CloudFormation::HookTypeConfig Handler returned status FAILED: Error occurred during operation 'One of TypeConfigurationArn or (TypeName and Type) or (TypeArn) should be specified'. (HandlerErrorCode: GeneralServiceException, RequestToken: dd442f50-c193-424c-bd44-af6b848e8174)
An error occurred (HandlerInternalFailureException) when calling the ListResources operation (reached max retries: 2): AWS::CloudFormation::HookDefaultVersion Handler returned status FAILED: The input should contain either ARN or Type Name (Service: CloudFormation, Status Code: 400, Request ID: 316d3467-aeac-4fc3-bfbe-4feda3cedb77, Extended Request ID: null) (HandlerErrorCode: InternalFailure, RequestToken: d10d81b4-64a8-4dc7-95db-eba05e3c3f2d)
An error occurred (AccessDeniedException) when calling the ListResources operation: AWS::CloudFormation::HookVersion Handler returned status FAILED: The security token included in the request is invalid (Service: CloudFormation, Status Code: 403, Request ID: 962ae1ba-3b70-4b45-9676-3e33adc549df, Extended Request ID: null) (HandlerErrorCode: AccessDenied, RequestToken: eb6e283f-6d07-41b1-b1c4-81a6e1d5c8f4)

[benbridts]

It's a bit less if you first filter out

• the resource types that are not supported by CloudControl API
• the resource types that require input (I haven't tested those if the input has to come from a broken resource):

(see here for my work-in-progress list)

And here is some python code to limit the resource types to the ones supported by CloudControl API:

def list_all_resource_types():
    """List the resource types that we can use with CloudControlApi."""
    # Supported types are FULLY_MUTABLE or IMMUTABLE and PUBLIC or PRIVATE
    for pt in ["FULLY_MUTABLE", "IMMUTABLE"]:
        kwargs = {"ProvisioningType": pt, "DeprecatedStatus": "LIVE", "Type": "RESOURCE"}
        # Public - return AWS types
        for page in cfn.get_paginator("list_types").paginate(
            Visibility="PUBLIC", Filters={"Category": "AWS_TYPES"}, **kwargs
        ):
            yield from (x["TypeName"] for x in page["TypeSummaries"])
        # Public - return activated types
        for page in cfn.get_paginator("list_types").paginate(
            Visibility="PUBLIC", Filters={"Category": "ACTIVATED"}, **kwargs
        ):
            yield from (x["TypeName"] for x in page["TypeSummaries"])
        # Private - return all types
        for page in cfn.get_paginator("list_types").paginate(Visibility="PRIVATE", **kwargs):
            yield from (x["TypeName"] for x in page["TypeSummaries"])

[PatMyron]

the resource types that are not supported by CloudControl API

assumed these would all return the UnsupportedActionException I'm catching

---

the resource types that require input

Should any input be required to list resources? I'd argue it could be a better experience if input wasn't required

Reasons I've seen involve resource type hierarchies, but the list handler itself could traverse listing the parent resource type if required to list without explicit input

---

Regardless, some of the list handlers with required inputs seem like they could have reasonable defaults:

Missing Or Invalid ResourceModel property in AWS::QuickSight::* list handler request input. Required property: [AwsAccountId]

[benbridts]

Should any input be required to list resources? I'd argue it could be a better experience if input wasn't required

I agree, although it does come with some extra complexities:

• Are there resources with multiple parents, or where the parent also needs to resolve parents?
• What effect does this have on the required permissions?

Currently the contract is not saying something either way:

A list handler MAY accept a set of properties conforming to the shape of the resource schema as filter criteria.

[MalikAtalla-AWS]

Thanks for raising this issue, Ben. I'm not able to reproduce the error message you're seeing. The behaviour has probably changed since you reported the problem. For ModuleDefaultVersion, the behaviour looks correct to me today as far as I can tell. For ModuleVersion the responses are currently not correct, so we decided to disable to ListHandler for now.

[benbridts]

This does indeed seems fixed for this resource, thanks!

[benbridts]

@MalikAtalla-AWS I can open a new issue too, but I still get this error for AWS::CloudFormation::HookVersion (and other resources)

~$ aws cloudcontrol list-resources --type-name AWS::CloudFormation::HookVersion

An error occurred (AccessDeniedException) when calling the ListResources operation: AWS::CloudFormation::HookVersion Handler returned status FAILED: The security token included in the request is invalid (Service: CloudFormation, Status Code: 403, Request ID: 313c850d-d5de-4565-85fc-9b814e6232e6) (HandlerErrorCode: AccessDenied, RequestToken: 1e98c732-3b12-44d2-a66e-1379ba587494)

[MalikAtalla-AWS]

Oh you're right. Maybe a new issue that mentions Hooks in the title would be clearer. I can create one and then ping the Hooks team. Thanks @benbridts

[MalikAtalla-AWS]

I have created a separate issue for the Hooks resources here -> https://github.com/aws-cloudformation/aws-cloudformation-resource-providers-cloudformation/issues/113

[benbridts]

Thanks!

as a tangent, all of these also give me a security token exception:

    "AWS::AppSync::DomainName",
    "AWS::Chatbot::MicrosoftTeamsChannelConfiguration",
    "AWS::CloudFormation::HookVersion",
    "AWS::CodeGuruReviewer::RepositoryAssociation",
    "AWS::Evidently::Segment",
    "AWS::MediaTailor::PlaybackConfiguration",
    "AWS::Pipes::Pipe",
    "AWS::RolesAnywhere::TrustAnchor",
    "AWS::Route53Resolver::FirewallDomainList",
    "AWS::Route53Resolver::FirewallRuleGroup",
    "AWS::Route53Resolver::FirewallRuleGroupAssociation",

    "AWS::SageMaker::DataQualityJobDefinition",
    "AWS::SageMaker::ModelBiasJobDefinition",
    "AWS::SageMaker::ModelExplainabilityJobDefinition",
    "AWS::SageMaker::ModelQualityJobDefinition",

[wkexiang]

Looks like this issue is not resolved. We are still seeing AccessDeniedException for list handler for HookVersion. I've also checked some of the resources mentioned in the previous comment and they also suffer from similar error. Can we please get someone to look into this? It looks to me that the issue is related to the framework but not individual resource handler.

[wkexiang]

The root cause is that HookVersion misses the permission of a specific API call in the handler. I assume the same thing happens to all other resources listed above. Hooks team will fix HookVersion's permission by end of July 2023.