Amplify Examples: AuthPostConfirmation + CDN (storage + cloudfront)

[armenr]

Is this related to a new or existing framework?

No response

Is this related to a new or existing API?

Authentication, Storage

Is this related to another service?

No response

Describe the feature you'd like to request

Two "features" come to mind that I think would be high-value for customers:

1. PostConfirmation Function Enhancement

It's extremely common for users of Amplify to want to insert a Cognito user into a table like User or Users after a user has successfully confirmed their signup.

Currently: When a user adds a PostConfirmation function, that function shows them how to catch cognito user being confirmed, and automatically add that user to a standard/default userpool group

Desirement: When an Amplify customer uses the CLI to add or update their Auth category, and they add a PostConfirmation function, it would be good to also boilerplate the code they would require in order to not only add that user to a cognito group, but also how to add that user to a table in Dynamo via AppSync

How I figured it out

I was able to figure this out in the following way:

1. Added a postconfirmation function using the auth category cli flow
2. Took a wild guess that maybe there's an example of how to interact with AppSync via a lambda by going through the Function category cli flow
3. It turns out, I was right: -->

? Select which capability you want to add: Lambda function (serverless function)
? Provide an AWS Lambda function name: doAppSyncStuffEasilyBecauseWeCan
? Choose the runtime that you want to use: NodeJS
? Choose the function template that you want to use: AppSync - GraphQL API request (with IAM)

✅ Available advanced settings:
- Resource access permissions
- Scheduled recurring invocation
- Lambda layers configuration
- Environment variables configuration
- Secret values configuration

? Do you want to configure advanced settings? Yes
? Do you want to access other resources in this project from your Lambda function? Yes
? Select the categories you want this function to have access to. api
? Api has 2 resources in this project. Select the one you would like your Lambda to access redactedprojectV2Gushak
? Select the operations you want to permit on redactedprojectV2Gushak Query, Mutation

You can access the following resource attributes as environment variables from your Lambda function
    API_REDACTEDPROJECTV2GUSHAK_GRAPHQLAPIENDPOINTOUTPUT
    API_REDACTEDPROJECTV2GUSHAK_GRAPHQLAPIIDOUTPUT
    API_REDACTEDPROJECTV2GUSHAK_GRAPHQLAPIKEYOUTPUT
    ENV
    REGION
? Do you want to invoke this function on a recurring schedule? No
? Do you want to enable Lambda layers for this function? No
? Do you want to configure environment variables for this function? No
? Do you want to configure secret values this function can access? No
✔ Choose the package manager that you want to use: · PNPM
? Do you want to edit the local lambda function now? No
✅ Successfully added resource doAppSyncStuffEasilyBecauseWeCan locally.

Note: I wish this aspect of the CLI was documented clearly, or highlighted somewhere...it would save so many people SO much time.

We originally spent DAYS trying to figure out our own approach for this...it involved hacking on a custom lambda that wrote raw data directly to Dynamo, as well as figuring out how to try to automatically add and attach IAM roles to the lambda that would permit access to that DynamoDB table + environment variables to specify the table name itself (which is hard...because table names are always randomly generated, they are not straightforward, so each time you create a new backend, you have to first ship and deploy the API, then go get the table name, then set that as an env var).

^^ As you can see, this is a giant pain.

4. Then I had to go and update my postConfirmation function to match the config/flow of the Lambda I just created in the Function category flow

❯ amplify update function
? Select the Lambda function you want to update redactedprojectV2DevelopAuthPostConfirmation
General information
- Name: redactedprojectV2DevelopAuthPostConfirmation
- Runtime: nodejs

Resource access permission
- redactedprojectV2Gushak (Query, Mutation)

Scheduled recurring invocation
- Not configured

Lambda layers
- Not configured

Environment variables:
- Not configured

Secrets configuration
- Not configured

? Which setting do you want to update? Resource access permissions
? Select the categories you want this function to have access to. api
? Api has 2 resources in this project. Select the one you would like your Lambda to access redactedprojectV2Gushak
? Select the operations you want to permit on redactedprojectV2Gushak Query, Mutation

You can access the following resource attributes as environment variables from your Lambda function
    API_REDACTEDPROJECTV2GUSHAK_GRAPHQLAPIENDPOINTOUTPUT
    API_REDACTEDPROJECTV2GUSHAK_GRAPHQLAPIIDOUTPUT
    API_REDACTEDPROJECTV2GUSHAK_GRAPHQLAPIKEYOUTPUT
? Do you want to edit the local lambda function now? No

5. Then I basically copy/pasted the code from the doAppSyncStuffEasilyBecauseWeCan lambda into my existing REDACTEDPROJECTV2DevelopAuthPostConfirmation function, and essentially modified the graphql mutation for inserting the user...(also, we customize our functions so that we can ship them in TypeScript...happy to share our implementation for that as well...it involves using rollup to emit a single .js file)

// file: amplify/backend/function/REDACTEDPROJECTV2DevelopAuthPostConfirmation/lib/add-to-users-via-api.ts
/* Amplify Params - DO NOT EDIT
  API_REDACTEDPROJECTV2GUSHAK_GRAPHQLAPIENDPOINTOUTPUT
  API_REDACTEDPROJECTV2GUSHAK_GRAPHQLAPIIDOUTPUT
  API_REDACTEDPROJECTV2GUSHAK_GRAPHQLAPIKEYOUTPUT
  ENV
  REGION
Amplify Params - DO NOT EDIT */

import crypto from '@aws-crypto/sha256-js';
import { defaultProvider } from '@aws-sdk/credential-provider-node';
import { SignatureV4 } from '@aws-sdk/signature-v4';
import { HttpRequest } from '@aws-sdk/protocol-http';
import { PostConfirmationTriggerHandler } from 'aws-lambda';
import fetch from 'cross-fetch';

const appsyncUrl = process.env.API_REDACTEDPROJECTV2GUSHAK_GRAPHQLAPIENDPOINTOUTPUT;
const AWS_REGION = process.env.AWS_REGION || 'us-east-1';
const { Sha256 } = crypto;

export const addUsersViaAPIHandler: PostConfirmationTriggerHandler = async (event) => {
  if (event.request.userAttributes.sub) {
    console.log(`EVENT: ${JSON.stringify(event.request.userAttributes.sub)}`)
  }

  const queryVars = {
    birthdate: event.request.userAttributes.birthdate,
    emailAddress: event.request.userAttributes.email,
    firstName: event.request.userAttributes.given_name,
    id: event.request.userAttributes.sub,
    lastName: event.request.userAttributes.family_name,
    owner: event.request.userAttributes.sub,
    phoneNumber: event.request.userAttributes.phone_number,
  };

  console.log(`queryVars: ${JSON.stringify(queryVars)}`)
  console.log(queryVars)

  // specify GraphQL request POST body or import from an extenal GraphQL document
  const createUserBody = {
    query: `
        mutation CreateUser($input: CreateUserInput!) {
          createUser(input: $input) {
            birthdate
            createdAt
            emailAddress
            firstName
            id
            lastName
            owner
            phoneNumber
            updatedAt
          }
        }
      `,
    operationName: 'CreateUser',
    variables: {
      input: {
        birthdate: event.request.userAttributes.birthdate,
        emailAddress: event.request.userAttributes.email,
        firstName: event.request.userAttributes.given_name,
        id: event.request.userAttributes.sub,
        lastName: event.request.userAttributes.family_name,
        owner: event.request.userAttributes.sub,
        phoneNumber: event.request.userAttributes.phone_number,
      },
    },
  };

  // parse URL into its portions such as hostname, pathname, query string, etc.
  const url = new URL(appsyncUrl);

  // set up the HTTP request
  const request = new HttpRequest({
    hostname: url.hostname,
    path: url.pathname,
    body: JSON.stringify(createUserBody),
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      host: url.hostname,
    },
  });

  // create a signer object with the credentials, the service name and the region
  const signer = new SignatureV4({
    credentials: defaultProvider(),
    service: 'appsync',
    region: AWS_REGION,
    sha256: Sha256,
  });

  try {
    // sign the request and extract the signed headers, body and method
    const { headers, body, method } = await signer.sign(request);

    // send the signed request and extract the response as JSON
    const response = await fetch(appsyncUrl, { headers, body, method });

    if (!response.ok) {
      throw new Error(`HTTP error! status: ${response.status}`);
    }

    const result = await response.json();
    console.log(`RESULT: ${JSON.stringify(result)}`);

    // Cognito User Pool expects the event object to be returned from this function
    return event;
  } catch (error) {
    console.error('An error occurred:', error);
    throw error;
  }
};

6. Then import all this into the main index file, and make sure it gets invoked sequentially (after the user is successfully added to the userPool group first)

/**
 * @fileoverview
 *
 * This CloudFormation Trigger creates a handler which awaits the other handlers
 * specified in the `MODULES` env var, located at `./${MODULE}`.
 */

import { PostConfirmationTriggerHandler } from 'aws-lambda';
import { addUsersViaAPIHandler } from './add-to-users-via-api';
import { addToGroupHandler } from './add-to-group';

/**
 * The names of modules to load are stored as a comma-delimited string in the
 * `MODULES` env var.
 */
// const moduleNames = (process.env.MODULES || '').split(',');

/**
 * This async handler iterates over the given modules and awaits them.
 *
 * @see https://docs.aws.amazon.com/lambda/latest/dg/nodejs-handler.html#nodejs-handler-async
 *
 */
export const handler: PostConfirmationTriggerHandler = async (event, context, callback) => {
  /**
   * Instead of naively iterating over all handlers, run them concurrently with
   * `await Promise.all(...)`. This would otherwise just be determined by the
   * order of names in the `MODULES` var.
   */

  try {
    await addToGroupHandler(event, context, callback);
    await addUsersViaAPIHandler(event, context, callback);
  } catch (error) {
    console.error(error)
    return error
  }

  return event
};

Final note on this one: Frankly, even if the code is not actually boiler-plated for you, if there was just a section in the documentation under the "Auth" category which walks you through doing this as a "Common Use-Case" or "Common Example" in an "Amplify-centric" way...even THAT would probably be high-value for customers.

2. An example of adding a custom CDN for Storage category

It's extremely common for users of Amplify to want to add some kind of CDN in front of their Storage. S3 + CloudFront is an extremely ubiquitous pattern...in fact, it's pretty much table stakes in any architecture.

It's been a time-consuming battle for us to figure out how to correctly add and then use a custom CDK resource which provides us with a fast CDN for serving files from S3 that have been uploaded into the app.

I'm still stuck trying to figure out how to protect files that were uploaded "privately" (only for the eyes of a user, or a userPool group), VS files available for all logged in users of the platform.

...I've seen countless posts across the web (StackOverflow, etc) where people basically attempt to implement this pattern, or ask for help to implement this pattern, or give up and abandon Amplify because they don't have a reference implementation or starting point for implementing this pattern.

I realize that in Gen2, this is likely much more intuitive or figure-out-able, but in gen1, this is pretty hard 😢 . It adds to the adoption curve being unfriendly for new users.

Describe the solution you'd like

1. Straightforward scaffold, example code, or at least example documentation with some reference code for having a PostConfirmation function that talks to AppSync and inserts a confirmed user for you, into your DB
2. Straightforward scaffold, example code, or at least example/reference code + documentation that allows an Amplify customer to correctly and straightforward-ly add a CDN (cloudfront) with/alongside their Storage category.

Describe alternatives you've considered

Ditching Amplify altogether, and going pure CDK (which would suck, because there's lots of goodness in the amplify cli + amplify categories based workflows).

Additional context

No response

Is this something that you'd be interested in working on?

• [ ] 👋 I may be able to implement this feature request
• [ ] ⚠️ This feature might incur a breaking change

[cwomack]

Hello again, @armenr 👋. This is some great feedback on both the documentation improvement side to save others time that want to implement some postConfirmation actions on user sign-up, as well as the feature request to have more "out of the box" support for CloudFront CDN to work seamlessly with the Storage category.

I'll review these with the team and let you know once have updates on either front! Thank you for taking the time to create this issue and provide so much information and context.

[armenr]

Thanks @cwomack ! If there's some way to help contribute or participate, I'd be glad to take some work and run with it myself.

If not, still grateful for the attention and consideration. Happy to provide any further feedback or answer any other questions. As you've probably noticed, I really (weirdly) love working with Amplify...a lot.

[armenr]

As an example: https://github.com/aws-amplify/amplify-cli/issues/1910

Update on the S3/CDN thing:

I have gotten fairly far with the custom CDK resource pattern, in a relatively short period of time. Here's what I've got so far:

import * as cdk from "aws-cdk-lib";
import * as AmplifyHelpers from "@aws-amplify/cli-extensibility-helper";
import { AmplifyDependentResourcesAttributes } from "../../types/amplify-dependent-resources-ref";
import { Construct } from "constructs";
import { CloudFrontToS3 } from '@aws-solutions-constructs/aws-cloudfront-s3';
import * as s3 from 'aws-cdk-lib/aws-s3';

// TODO: Add a certificate and DNS to make things nice and prod-friendly
// import * as acm from 'aws-cdk-lib/aws-certificatemanager'
// import * as route53 from 'aws-cdk-lib/aws-route53'

export class cdkStack extends cdk.Stack {
  constructor(
    scope: Construct,
    id: string,
    props?: cdk.StackProps,
    amplifyResourceProps?: AmplifyHelpers.AmplifyResourceProps,
  ) {
    super(scope, id, props);
    /* Do not remove - Amplify CLI automatically injects the current deployment environment in this input parameter */
    new cdk.CfnParameter(this, "env", {
      type: "String",
      description: "Current Amplify CLI env name",
    });

    // const amplifyProjectInfo = AmplifyHelpers.getProjectInfo();
    const dependencies: AmplifyDependentResourcesAttributes = AmplifyHelpers.addResourceDependency(this,
      amplifyResourceProps.category,
      amplifyResourceProps.resourceName,
      [{
        category: "storage",
        resourceName: "REDACTEDPROJECTV2DevelopStorage"
      }]
    );

    const bucketName = cdk.Fn.ref(dependencies.storage.REDACTEDPROJECTV2DevelopStorage.BucketName)
    const bucket = s3.Bucket.fromBucketName(this, bucketName, bucketName);

    const customCDN = new CloudFrontToS3(this, 'app-cdn', {
      existingBucketObj: bucket,
    });

    new cdk.CfnOutput(this, 'CDNURL', {
      value: customCDN.cloudFrontWebDistribution.domainName,
      description: 'The URL of the custom CloudFront distribution',
    });
  }
}

package.json:

{
  "dependencies": {
    "@aws-amplify/cli-extensibility-helper": "^3.0.24",
    "@aws-solutions-constructs/aws-cloudfront-s3": "^2.48.0",
    "aws-cdk-lib": "~2.118.0",
    "constructs": "^10.3.0"
  },
  "description": "",
  "devDependencies": {
    "typescript": "^5.3.3"
  },
  "name": "custom-resource",
  "resolutions": {
    "aws-cdk-lib": "~2.118.0"
  },
  "scripts": {
    "build": "tsc",
    "test": "echo \"Error: no test specified\" && exit 1",
    "watch": "tsc -w"
  },
  "version": "1.0.0"
}

Caveats:

1. You need to force the resolution of the cdk-lib, else you get conflicting types between different versions of the cdk-lib as a transitive dependency
2. You need to add additional IAM policies to your amplify deployment role or user, which grants necessary permissions on cloudfront + s3, so that the CDN + the access logs can be set up correctly with a vanilla amplify push

Now, I'm here:

My guess is that I need to figure out how to generate and handle signed URLs for authenticated users of the app.

[armenr]

Follow-up: I did already dig through documentation + github issues...and I tried to ask this in the Discord as well...

But how can we essentially synth a custom CDK resource we added with the CLI? Is there any possible way to do this?

[cwomack]

@armenr, I'm actually going to transfer this issue to the amplify-backend repo to get you better assistance on this. We already have the Storage category aspect of this feature-request being tracked in issue aws-amplify/amplify-js#9418!

[armenr]

@cwomack - thanks! :)

[armenr]

Things I tried that didn't work:

• Add a full custom resource stack via CDK that:
  • Consumes the existing S3 bucket from our Storage category via AmplifyHelpers.addResourceDependency()
  • Uses the existing S3 bucket it consumed to set up CloudFront + S3 CDN scheme withCloudFrontToS3 CDK construct
  • Creates private/public key pair and drops all that off in SecretsManager
  • Adds the publicKeyConfig to CloudFront
  • Makes the secretKey available for a Lambda "signer" function to use, for generating signed URLs to the assets in the bucket, which would then go through the CDN's URL

All that went without an issue, and I was very excited.

Then, with this CDK custom stack approach, I hit a massive roadblock:

• I declared a Lambda Function + an API Gateway for that function
• I created an apigateway.CfnAuthorizer resource
• I created an api.root.addResource('signed-url') resource
• Then I used signedUrlResource.addMethod to create a GET route on the API, and passed in my authorizer + AuthorizationType.COGNITO
• ALL of that went fine...but then...
• After a lot of pain and a few wasted hours, I learned that Amplify does not allow you to deploy Lambdas at all via custom CDK resources (deployment key/bucket not found)

🤷 So there went about 6 hours worth of trail & error work.

So then I tried:

• Creating my signer Lambda via Amplify CLI Lambda Category
• Consuming that existing Lambda via AmplifyHelpers.addResourceDependency() in my CDK custom stack, rather than creating the lambda directly in my CDK stack
• Once I had access to the Lambda in CDK, I wanted to modify the existing Lambda to dynamically add/inject the privateKeySecret.secretArn from SSM + the customCDN.cloudFrontWebDistribution.domainName as environment variables on the Lambda...but I couldn't.
  • I attempted to do that via the aws-sdk via the sdk's Lambda lib's updateFunctionConfiguration method.
  • This, too, was a doomed attempt, since using the SDK this way, inside the CDK stack is also doomed.

I couldn't have the lambda pull the outputs of the CDK stack in as its own dependency, and consume those as outputs from the custom CDK stack, because the CDK stack would also depend on the Lambda vi the CDK stack's REST API Gateway + authorizer resource...and this would become a circular dependency fiasco.

The real problem here is that you want your entire backend to be dynamic/portable. Ideally, anyone with a professional, production-ready workflow would simply be able to check out many different copies of their amplify backend and simply push the backend, and see everything build...without manual intervention.

That means:

• No manually generated or added secrets
• No manually retrieved ARNs or manually attached/added permissions or policies on any resources in their stack, after automated deployment.

I am now attempting a different approach that combines:

1. An AWS CDK custom resource (uses Storage category bucket, creates keys, creates cloudfront CDN, sets that up, spits out useful outputs)
2. An Amplify API category REST + Lambda setup with the URL signer Lambda in there
3. amplify override api to then attempt to add the authorizer on the API gateway there
4. Consuming the outputs from the CDK stack uni-directionally from the CDK stack, into the Amplify CLI-created API Gateway + lambda signer function

Even in this case, there's still a problem I'm considering...which is: The CDK custom resource would have to have already been deployed/existent, when the amplify cli-generated API Gateway + Lambda are created, because the Lambda would be consuming the outputs from the custom CDK stack...and on a "first push" of a newly created/checked out backend...those won't exist or be present 😢

Would that be solvable by adding a dependsOn to the Lambda, and having it depend on the cust CDK resource stack's output(s)? That should ensure that the CDK stack is created/executed further up in the resource graph, right?

I gotta tell you, I've been doing this in my free time outside of work...and it's been many hours of trial & error, just to figure out a "clean" pattern that doesn't necessitate manual intervention or direct knowledge of any unique resource...in other words, a "generic" and "portable" implementation.

I'll report back with results/findings.

[armenr]

The only other thing I can think of is to do all of this totally and purely outside of Amplify, in a separate CDK project that lives in the same repo next to everything else.

Then, we'd take a similar approach to what Heitor Lessa did here, in this repo's amplify.yml -- https://github.com/aws-samples/aws-serverless-airline-booking/blob/archive/amplify.yml

Basically deploy all the amplify stuff, then use jq to export and push necessary outputs from the Amplify backend into parameter store or SSM...and then deploy the separate CDK project inside the repo, and have the CDK project reference those parameters or secrets at their expected keys/locations in parameter store/SSM.

Just another possible set of acrobatics I'm going to attempt.

[josefaidt]

Hey @armenr :wave: thank you for raising this and providing the details despite the hurdles and roadblocks. Would you be open to hopping on a quick call to chat about your experiences?

[armenr]

@josefaidt - Absolutely :)

I pinged you on Discord privately.

[armenr]

Once you know and understand the plumbing of the framework and its constraints, implementation is actually not so bad.

The part of the process that consumes lots of trial&error time and frustrates the customer is the lack of reference implementation and no clear indication of the limitations, caveats, or plumbing.

I have a fully working setup now, including private/public keys for signed URLs against protected objects in S3, with an Amplify-provided lambda that generates signed URLs via CloudFront.

It's fully portable. You can checkout to a fresh backend, and just amplify push without worrying about doing anything by hand or hard-coding anything in the code or the AWS console. You don't have to use the cli to set any secrets or variables either. It's zero-friction once you implement it the first time.

This is likely a really good use-case for a plugin-style approach, I think. No clue how it factors into the gen2 approach, however. I'd guess it's even easier in gen2 context.

I might take a shot at writing a plugin for it...

As is, it's not totally copy/paste-able as a documentation-provided solution from a DX perspective, especially for newcomers.

🚀