Open armenr opened 7 months ago
Hello again, @armenr 👋. This is some great feedback on both the documentation improvement side to save others time that want to implement some postConfirmation actions on user sign-up, as well as the feature request to have more "out of the box" support for CloudFront CDN to work seamlessly with the Storage category.
I'll review these with the team and let you know once have updates on either front! Thank you for taking the time to create this issue and provide so much information and context.
Thanks @cwomack ! If there's some way to help contribute or participate, I'd be glad to take some work and run with it myself.
If not, still grateful for the attention and consideration. Happy to provide any further feedback or answer any other questions. As you've probably noticed, I really (weirdly) love working with Amplify...a lot.
As an example: https://github.com/aws-amplify/amplify-cli/issues/1910
I have gotten fairly far with the custom CDK resource pattern, in a relatively short period of time. Here's what I've got so far:
import * as cdk from "aws-cdk-lib";
import * as AmplifyHelpers from "@aws-amplify/cli-extensibility-helper";
import { AmplifyDependentResourcesAttributes } from "../../types/amplify-dependent-resources-ref";
import { Construct } from "constructs";
import { CloudFrontToS3 } from '@aws-solutions-constructs/aws-cloudfront-s3';
import * as s3 from 'aws-cdk-lib/aws-s3';
// TODO: Add a certificate and DNS to make things nice and prod-friendly
// import * as acm from 'aws-cdk-lib/aws-certificatemanager'
// import * as route53 from 'aws-cdk-lib/aws-route53'
export class cdkStack extends cdk.Stack {
constructor(
scope: Construct,
id: string,
props?: cdk.StackProps,
amplifyResourceProps?: AmplifyHelpers.AmplifyResourceProps,
) {
super(scope, id, props);
/* Do not remove - Amplify CLI automatically injects the current deployment environment in this input parameter */
new cdk.CfnParameter(this, "env", {
type: "String",
description: "Current Amplify CLI env name",
});
// const amplifyProjectInfo = AmplifyHelpers.getProjectInfo();
const dependencies: AmplifyDependentResourcesAttributes = AmplifyHelpers.addResourceDependency(this,
amplifyResourceProps.category,
amplifyResourceProps.resourceName,
[{
category: "storage",
resourceName: "REDACTEDPROJECTV2DevelopStorage"
}]
);
const bucketName = cdk.Fn.ref(dependencies.storage.REDACTEDPROJECTV2DevelopStorage.BucketName)
const bucket = s3.Bucket.fromBucketName(this, bucketName, bucketName);
const customCDN = new CloudFrontToS3(this, 'app-cdn', {
existingBucketObj: bucket,
});
new cdk.CfnOutput(this, 'CDNURL', {
value: customCDN.cloudFrontWebDistribution.domainName,
description: 'The URL of the custom CloudFront distribution',
});
}
}
package.json:
{
"dependencies": {
"@aws-amplify/cli-extensibility-helper": "^3.0.24",
"@aws-solutions-constructs/aws-cloudfront-s3": "^2.48.0",
"aws-cdk-lib": "~2.118.0",
"constructs": "^10.3.0"
},
"description": "",
"devDependencies": {
"typescript": "^5.3.3"
},
"name": "custom-resource",
"resolutions": {
"aws-cdk-lib": "~2.118.0"
},
"scripts": {
"build": "tsc",
"test": "echo \"Error: no test specified\" && exit 1",
"watch": "tsc -w"
},
"version": "1.0.0"
}
Caveats:
amplify push
Now, I'm here:
My guess is that I need to figure out how to generate and handle signed URLs for authenticated users of the app.
Follow-up: I did already dig through documentation + github issues...and I tried to ask this in the Discord as well...
But how can we essentially synth
a custom CDK resource we added with the CLI? Is there any possible way to do this?
@armenr, I'm actually going to transfer this issue to the amplify-backend
repo to get you better assistance on this. We already have the Storage category aspect of this feature-request being tracked in issue aws-amplify/amplify-js#9418!
@cwomack - thanks! :)
Things I tried that didn't work:
AmplifyHelpers.addResourceDependency()
CloudFrontToS3
CDK constructAll that went without an issue, and I was very excited.
Then, with this CDK custom stack approach, I hit a massive roadblock:
apigateway.CfnAuthorizer
resourceapi.root.addResource('signed-url')
resourcesignedUrlResource.addMethod
to create a GET route on the API, and passed in my authorizer + AuthorizationType.COGNITO
deployment key/bucket not found
)🤷 So there went about 6 hours worth of trail & error work.
So then I tried:
AmplifyHelpers.addResourceDependency()
in my CDK custom stack, rather than creating the lambda directly in my CDK stackprivateKeySecret.secretArn
from SSM + the customCDN.cloudFrontWebDistribution.domainName
as environment variables on the Lambda...but I couldn't.
aws-sdk
via the sdk's Lambda lib's updateFunctionConfiguration
method. I couldn't have the lambda pull the outputs of the CDK stack in as its own dependency, and consume those as outputs from the custom CDK stack, because the CDK stack would also depend on the Lambda vi the CDK stack's REST API Gateway + authorizer resource...and this would become a circular dependency fiasco.
The real problem here is that you want your entire backend to be dynamic/portable. Ideally, anyone with a professional, production-ready workflow would simply be able to check out many different copies of their amplify backend and simply push the backend, and see everything build...without manual intervention.
That means:
I am now attempting a different approach that combines:
amplify override api
to then attempt to add the authorizer on the API gateway thereEven in this case, there's still a problem I'm considering...which is: The CDK custom resource would have to have already been deployed/existent, when the amplify cli-generated API Gateway + Lambda are created, because the Lambda would be consuming the outputs from the custom CDK stack...and on a "first push" of a newly created/checked out backend...those won't exist or be present 😢
Would that be solvable by adding a dependsOn
to the Lambda, and having it depend on the cust CDK resource stack's output(s)? That should ensure that the CDK stack is created/executed further up in the resource graph, right?
I gotta tell you, I've been doing this in my free time outside of work...and it's been many hours of trial & error, just to figure out a "clean" pattern that doesn't necessitate manual intervention or direct knowledge of any unique resource...in other words, a "generic" and "portable" implementation.
I'll report back with results/findings.
The only other thing I can think of is to do all of this totally and purely outside of Amplify, in a separate CDK project that lives in the same repo next to everything else.
Then, we'd take a similar approach to what Heitor Lessa did here, in this repo's amplify.yml
-- https://github.com/aws-samples/aws-serverless-airline-booking/blob/archive/amplify.yml
Basically deploy all the amplify stuff, then use jq
to export and push necessary outputs from the Amplify backend into parameter store or SSM...and then deploy the separate CDK project inside the repo, and have the CDK project reference those parameters or secrets at their expected keys/locations in parameter store/SSM.
Just another possible set of acrobatics I'm going to attempt.
Hey @armenr :wave: thank you for raising this and providing the details despite the hurdles and roadblocks. Would you be open to hopping on a quick call to chat about your experiences?
@josefaidt - Absolutely :)
I pinged you on Discord privately.
Once you know and understand the plumbing of the framework and its constraints, implementation is actually not so bad.
The part of the process that consumes lots of trial&error time and frustrates the customer is the lack of reference implementation and no clear indication of the limitations, caveats, or plumbing.
I have a fully working setup now, including private/public keys for signed URLs against protected objects in S3, with an Amplify-provided lambda that generates signed URLs via CloudFront.
It's fully portable. You can checkout to a fresh backend, and just amplify push without worrying about doing anything by hand or hard-coding anything in the code or the AWS console. You don't have to use the cli to set any secrets or variables either. It's zero-friction once you implement it the first time.
This is likely a really good use-case for a plugin-style approach, I think. No clue how it factors into the gen2 approach, however. I'd guess it's even easier in gen2 context.
I might take a shot at writing a plugin for it...
As is, it's not totally copy/paste-able as a documentation-provided solution from a DX perspective, especially for newcomers.
🚀
Is this related to a new or existing framework?
No response
Is this related to a new or existing API?
Authentication, Storage
Is this related to another service?
No response
Describe the feature you'd like to request
Two "features" come to mind that I think would be high-value for customers:
1. PostConfirmation Function Enhancement
It's extremely common for users of Amplify to want to insert a Cognito user into a table like
User
orUsers
after a user has successfully confirmed their signup.Currently: When a user adds a PostConfirmation function, that function shows them how to catch cognito user being confirmed, and automatically add that user to a standard/default userpool group
Desirement: When an Amplify customer uses the CLI to add or update their Auth category, and they add a PostConfirmation function, it would be good to also boilerplate the code they would require in order to not only add that user to a cognito group, but also how to add that user to a table in Dynamo via AppSync
How I figured it out
I was able to figure this out in the following way:
Note: I wish this aspect of the CLI was documented clearly, or highlighted somewhere...it would save so many people SO much time.
We originally spent DAYS trying to figure out our own approach for this...it involved hacking on a custom lambda that wrote raw data directly to Dynamo, as well as figuring out how to try to automatically add and attach IAM roles to the lambda that would permit access to that DynamoDB table + environment variables to specify the table name itself (which is hard...because table names are always randomly generated, they are not straightforward, so each time you create a new backend, you have to first ship and deploy the API, then go get the table name, then set that as an env var).
^^ As you can see, this is a giant pain.
doAppSyncStuffEasilyBecauseWeCan
lambda into my existingREDACTEDPROJECTV2DevelopAuthPostConfirmation
function, and essentially modified the graphql mutation for inserting the user...(also, we customize our functions so that we can ship them in TypeScript...happy to share our implementation for that as well...it involves using rollup to emit a single.js
file)index
file, and make sure it gets invoked sequentially (after the user is successfully added to the userPool group first)Final note on this one: Frankly, even if the code is not actually boiler-plated for you, if there was just a section in the documentation under the "Auth" category which walks you through doing this as a "Common Use-Case" or "Common Example" in an "Amplify-centric" way...even THAT would probably be high-value for customers.
2. An example of adding a custom CDN for Storage category
It's extremely common for users of Amplify to want to add some kind of CDN in front of their Storage. S3 + CloudFront is an extremely ubiquitous pattern...in fact, it's pretty much table stakes in any architecture.
It's been a time-consuming battle for us to figure out how to correctly add and then use a custom CDK resource which provides us with a fast CDN for serving files from S3 that have been uploaded into the app.
I'm still stuck trying to figure out how to protect files that were uploaded "privately" (only for the eyes of a user, or a userPool group), VS files available for all logged in users of the platform.
...I've seen countless posts across the web (StackOverflow, etc) where people basically attempt to implement this pattern, or ask for help to implement this pattern, or give up and abandon Amplify because they don't have a reference implementation or starting point for implementing this pattern.
I realize that in Gen2, this is likely much more intuitive or figure-out-able, but in gen1, this is pretty hard 😢 . It adds to the adoption curve being unfriendly for new users.
Describe the solution you'd like
Describe alternatives you've considered
Ditching Amplify altogether, and going pure CDK (which would suck, because there's lots of goodness in the amplify cli + amplify categories based workflows).
Additional context
No response
Is this something that you'd be interested in working on?