Best way to handle credentials for external services

[Giaco9]

Hi everyone, I'm implementing a custom resource provider that creates a Project into an external service. That service exposes a set of APIs that requires a token to authenticate the API call. My schema is:

{
    "typeName": "Company::ExternalService::Project",
    "description": "Create a project",
    "additionalProperties": false,
    "properties": {
        "ProjectName": {
            "description": "Name of the project",
            "type": "string",
            "pattern": "^[a-zA-Z0-9-_]{1,32}$"
        },
        "APIAccessToken": {
            "description": "API Access token to authenticate with the API",
            "type": "string"
        },
        "ProjectId": {
            "description": "ID of the new project",
            "type": "integer"
        },
        "CreationDate": {
            "description": "Creation date of the project",
            "type": "string"
        }
    },
    "required": [
        "ProjectName",
        "APIAccessToken"
    ],
    "readOnlyProperties": [
        "/properties/ProjectId",
        "/properties/CreationDate"
    ],
    "primaryIdentifier": [
        "/properties/ProjectId"
    ],
    "handlers": {
        "create": {
            "permissions":[]
        },
        "read": {
            "permissions": []
        },
        "delete": {
            "permissions": []
        },
        "list": {
            "permissions": []
        }
    }
}

To create a project I do the following:

Resources:
  ExternalServiceProject:
    Type: Company::ExternalService::Project
    Properties:
      ProjectName: NewProjectName
      APIAccessToken: <API Access Token From the secret store>

Ouputs:
  ProjectId:
     Value: !Ref ExternalServiceProject
  ProjectCreationDate:
      Value: !GetAtt ExternalServiceProject.CreationDate

The ExternalServiceProject physical resource id is "project$NewProjectName$<project-id-generated-by-the-external-service}" Adding the ProjectCreationDate output when I create the stack, the project is created successfully but the subsequent read API call doesn't receive the APIAccessToken to describe the project created. Without the APIAccessToken, I can't call the API to describe the project and get the creation date. The delete request to roll back the progress creation receives the API Access Token and it is able to delete the project correctly. Which is the best way to handle API credentials to external services? I don't want to put them as part of the physical resource id because is a secret, I don't want to put the Creation Date as part of the physical resource id because it doesn't scale as a solution.

$ cfn --version
cfn 0.1.13

Cloudformation cli python plugin version: 2.1.4 Python 3.7

Thank you!

[eduardomourar]

i believe it is the related to (or duplicated of) issue #584

[Giaco9]

@eduardomourar I think yes. Probably mine wants to address a more specific use case. Maybe there is an easier solution for this, but solving #584 would solve both issues

[PatMyron]

i believe it is the related to (or duplicated of) issue #584

I don't think this would solve ListHandler at least, which is not resource-specific and would also need external credentials

[carpnick]

The only way(I found) to solve this is to store the information and pull it from somewhere.
I solved this generically for my custom providers by creating a dynamodb library that supports create/update/delete/list/read handlers. Then depending on the call, I can read from the table the original input. This allowed me to guarantee getting original input in the read handler case. This doesnt help at all for import use case.

For the import/list use case: The workflow is fundamentally broken. If you ever need api credentials - you will need a way to put in original requested state. This allows you to input credentials, if required, and allows you to also do the list. One could argue you could use the Global account level values(like hooks work, see screen shot) But that seems like anti-pattern magic IMO. Not very user friendly and not expected IMO.

While I think import/list is important - if we cant answer the question of how a read handler is supposed to work with just the primary identifier (without a UI for creds) - It seems we are a bit of distance away from enabling truly generic import.

[benbridts]

I actually think that the per-region setting is better way to do this (that wasn't available at launch):

If you define the AccessKey (or the name of the parameter / secret that contains it) as part of the type configuration, CloudFormation will ask the consumer to provide that (per account/region) when activating the resource and send you that information on every invocation.

If you'd need multiple access keys, you could active the same type under multiple different names

The Schema docs say:

typeConfiguration A type configuration schema that defines any properties that the user must specify for all instances of the extension in a given account and Region. For example, if your extension needs to access a third-party web service, you can include a configuration schema for the user to specify their credentials for that service. [...] CloudFormation [...] saves this information at the Region level. [...]. Configurations are available to CloudFormation during all resource operations, including read and list events that don't explicitly involve a stack template.

There is also an example schema here

You can also look at (eg) the GitHub resource providers for an example: https://github.com/aws-ia/cloudformation-github-resource-providers

[carpnick]

Thanks for this @benbridts - This works with the single account use case(maybe). But in a multi-account scenario where you dont want to share credentials in all accounts since you have many teams involved - I dont know if I agree. I dont want to store the same credential in N accounts in N regions in secrets manager or parameter store. That seems like a huge anti-pattern to me.

[benbridts]

@carpnick Isn't that worse if you have to provide it via the template?

[carpnick]

@benbridts yes and no. The secret should be stored exactly once for external APIs if a user desires. That way if I need to rotate, I can change in one spot. Other use cases might be each AWS account gets 1 password. All of these options are user design decisions based on security requirements

I can use NoEcho on the sensitive parameter to pull in a parameter or secret defined in 1 AWS account. Or let cloudformation pull in secret or parameters appropriately.

All of this is making sweeping generalizations about security posture of customers and designing for the single AWS account use case when the prescriptive guidance is to migrate to a multi AWS account design. I think this design choice creates some friction for customers defaulting to multi-aws account use case.

[benbridts]

The secret should be stored exactly once for external APIs if a user desires. That way if I need to rotate, I can change in one spot. Other use cases might be each AWS account gets 1 password. All of these options are user design decisions based on security requirements

I think you can accomplish both if you use a reference to a (potentially shared) SecretsManager Secret

All of this is making sweeping generalizations about security posture of customers and designing for the single AWS account use case when the prescriptive guidance is to migrate to a multi AWS account design

I just don't see how having a NoEcho parameter is better than having account-level configuration. Either way exposes your credentials more than you'd like, and at least with the account-level configuration you don't have to tell it to everyone who needs to create a stack

[carpnick]

What if I dont want everyone to have access to the specific resource we are creating?

What if we have a shared responsibility model within a company - core infrastructure vs app infrastructure. I dont want my app teams to have access to the custom resources registered.

You could argue you can control that via fine grained IAM policies or Secret policies. But this adds alot of complexity on the implementation instead of just forcing creds in the stack. I would concede that it is "kicking the can down the road" because you still need to figure out how to put the stack parameters in properly. But at least then its a single implementation in dealing with credentials - not N implementations for each custom extension. If I plan on writing/using 30 extensions and 10 of them need credentials - all those fine grained policies would add complexity vs just requiring stack parameters

If you dont think this is realistic, at last company I created 40 Custom Resources using the old style framework and we had the same design discussion as here. We leaned toward stack parameters vs Custom Resource integrated with secrets via account level "registration" parameters instead for reasons above.

[carpnick]

I just don't see how having a NoEcho parameter is better than having account-level configuration. Either way exposes your credentials more than you'd like, and at least with the account-level configuration you don't have to tell it to everyone who needs to create a stack

Your point is true though - In some use cases I want to allow others to call the resource, and in some cases I dont. So whatever design exists needs to support both