Checkout out new env/auto-created bakend w/ continuous deployment breaks when using "prod" setup + signed URLs

[armenr]

Describe the bug Environment named "staging" has a video category resource set up. This includes a "production" setup with cloudfront + signed urls.

When checking out to a new env (amplify env add/amplify env checkout) "prod" and running amplify push, deploy fails.

Following resources failed

Resource Name: rCloudFrontPublicKeyifku7ykw (AWS::CloudFront::PublicKey)
Event Type: create
Reason: Resource handler returned message: "Resource of type 'AWS::CloudFront::PublicKey' with identifier 'streemr-staging-publickey-ifku7ykw' already exists." (RequestToken: 64b7932d-c06e-89ef-81ac-8daf422c8080, HandlerErrorCode: AlreadyExists)

To Reproduce Steps to reproduce the behavior:

1. Set up video category + production configuration + signed URLS in existing env
2. Add new env

   
   ╰─ amplify env add prod
   Note: It is recommended to run this command from the root of your app directory
   ? Do you want to use an existing environment? No
   ? Enter a name for the environment prod
   Using default provider  awscloudformation
   ? Select the authentication method you want to use: AWS profile

╰─ amplify status

Current Environment: prod

Category	Resource name	Operation	Provider plugin
Auth	gatekeep	Create	awscloudformation
Api	morpheuz	Create	awscloudformation
Video	streemr	Create	awscloudformation

╰─ amplify push

Following resources failed

Resource Name: rCloudFrontPublicKeyifku7ykw (AWS::CloudFront::PublicKey) Event Type: create Reason: Resource handler returned message: "Resource of type 'AWS::CloudFront::PublicKey' with identifier 'streemr-staging-publickey-ifku7ykw' already exists." (RequestToken: 64b7932d-c06e-89ef-81ac-8daf422c8080, HandlerErrorCode: AlreadyExists)

**Expected behavior**
Be able to have multiple environments

[armenr]

So, I did some digging. I must be missing something obvious/simple, but I think I'm almost there.

I copied /build/vod-helpers/CFDistribution.template --> to /custom/vod-helpers/CFDistribution.template

And this is what it looks like. Note that I added "env" as a parameter and also modified the Name and CallerReference for the AWS::CloudFront::PublicKey resource.

Description: CloudFront Distribution for output bucket

Parameters:
  env:
    Type: String
    Description: The environment name. e.g. Dev, Test, or Production
  pBucketUrl:
    Type: String
    Description: ProjectName
    Default: DefaultName
  pOriginAccessIdentity:
    Type: String
    Description: Policy for bucket
    Default: NA

  pProjectName:
    Type: String
    Description: Name for public key
    Default: PublicKey

Resources:
  rCloudFrontDist:
    Type: AWS::CloudFront::Distribution
    Properties:
      DistributionConfig:
        DefaultCacheBehavior:
          ForwardedValues:
            QueryString: false
            Cookies:
              Forward: none
            Headers:
              - "Origin"
              - "Access-Control-Request-Method"
              - "Access-Control-Request-Headers"
          AllowedMethods:
            - GET
            - HEAD
            - OPTIONS
          TargetOriginId: "vodS3Origin"
          ViewerProtocolPolicy: "allow-all"

          TrustedKeyGroups:
            - !Ref rCloudFrontKeyGroup

        Origins:
          - DomainName: !Ref pBucketUrl
            Id: vodS3Origin
            S3OriginConfig:
              OriginAccessIdentity: !Sub "origin-access-identity/cloudfront/${pOriginAccessIdentity}"
        Enabled: true
        PriceClass: PriceClass_All

  rCloudFrontPublicKey9v3fi11b:
    Type: AWS::CloudFront::PublicKey
    Properties:
      PublicKeyConfig:
        CallerReference:
          !Join ["-", ["araVOD", !Ref env, "publickey", "9v3fi11b"]]
        Name: !Join ["-", ["araVOD", !Ref env, "publickey", "9v3fi11b"]]
        EncodedKey: "-----BEGIN PUBLIC KEY-----\nREDACTED_KEY\n-----END PUBLIC KEY-----\n"
  rCloudFrontKeyGroup:
    Type: AWS::CloudFront::KeyGroup
    Properties:
      KeyGroupConfig:
        Name: !Sub "${pProjectName}-KeyGroup"
        Items:
          - !Ref rCloudFrontPublicKey9v3fi11b

Outputs:
  oPemId:
    Value: !Ref rCloudFrontPublicKey9v3fi11b
    Description: Pem Key Id

  oCFDomain:
    Value: !GetAtt rCloudFrontDist.DomainName
    Description: Domain for our videos

It however looks like env is missing at runtime. Anybody got any ideas or suggestions? :-\

CREATE_FAILED rCloudfrontDistribution AWS::CloudFormation::Stack Mon May 03 2021 10:55:40 GMT-0700 (Pacific Daylight Time) Parameters: [env] must have values

[armenr]

One other bit of source code I was able to find, but not completely wrap my head around is in vod-push.js

For the S3 input/output buckets, I see the following:

props.shared.bucketInput = `${nameDict.resourceName.toLowerCase()}-${projectDetails.localEnvInfo.envName}-input-${uuid}`.slice(0, 63);
props.shared.bucketOutput = `${nameDict.resourceName.toLowerCase()}-${projectDetails.localEnvInfo.envName}-output-${uuid}`.slice(0, 63);

However, for the CloudFront Distro, I see:

const secretName = `${props.shared.resourceName}-${projectDetails.localEnvInfo.envName}-pem-${uuid}`.slice(0, 63);
const rPublicName = `rCloudFrontPublicKey${uuid}`.slice(0, 63);
const publicKeyName = `${props.shared.resourceName}-${projectDetails.localEnvInfo.envName}-publickey-${uuid}`.slice(0, 63);

I'm happy to fiddle with the code, but just wondering if this is the right place to be looking?

[arturocanalda]

@armenr I just hit the same wall as you right now.

Let's see if we can fix this (otherwise I won't be able to deploy to production the work of the last 2 weeks :)

[armenr]

@arturocanalda

I'm working to find a solution. Will keep this thread updated.

[armenr]

@arturocanalda - I have what appears to be a working implementation here. I advise that you use it at your own risk, but I would sincerely appreciate if you could test with this a bit and let me know how it goes?

code is located here: https://github.com/armenr/amplify-video/tree/armenr/fix_multi_env

Please be sure to back up/git commit your current codebase, amplify folder, etc before testing.

Ideally, you could create a copy/duplicate of your existing project and amplify app so that we don't accidentally break or mangle any resources you currently have deployed, in your existing project

Steps For testing:

1. npm remove the existing amplify-video plugin globally/in your project (whichever approach you are using)
2. npm install -g https://github.com/armenr/amplify-video/\#armenr/fix_multi_env
3. Use the plugin as you would normally use it, and see if you're able to achieve the desired outcome (multiple backends with individual/unique video resource stacks for each)

You're welcome to commit on my fork, and please provide all the feedback you can. Thank you!

[arturocanalda]

Hi @armenr,

Thank you for sharing your fix :)

I tried it and added my comments in your PR.

[armenr]

@arturocanalda - Thanks for the feedback and information, and the time you spent on it!

I've updated the PR with questions for you, but which version of amplify & which version of node are you running?

[djsjr]

@armenr, I am having a related issue that I have a feeling can be solved similarly to how yours was. I cannot push multiple VOD resources in the same env (#208). The error is identical:

Following resources failed

Resource Name: rCloudFrontPublicKeydev4yxxo4g7 (AWS::CloudFront::PublicKey)
Event Type: create
Reason: Resource handler returned message: "Resource of type 'AWS::CloudFront::PublicKey' with identifier 'resourceName-dev-publickey-4yxxo4g7' already exists." (RequestToken: 1b35b02c-d44f-9178-9523-825120950616, HandlerErrorCode: AlreadyExists)

Any clue on how this could be fixed?