rails-lambda / lamby

🐑🛤 Simple Rails & AWS Lambda Integration
https://lamby.cloud
MIT License
602 stars 29 forks source link

Example template.yaml with cloudfront (Not an issue, just pasting for future searchers) #177

Closed jeremiahlukus closed 5 months ago

jeremiahlukus commented 5 months ago

It took me awhile to figure out how to get the function url in the cloudfront distro.

I want to share my config which does a few basic things

1. JobsQueue: Defines an SQS queue named JobsQueue with a visibility timeout of 300 seconds, used for handling job messages ie the WelcomeEmailJob.
2. RailsLambda: Creates a Lambda function for the Rails application, configured with VPC settings usually my databases are in a VPC therefore the lambda needs to be in it to communicate with it, environment variables, and policies for accessing SSM parameters and SQS.
3. PingHealthCheckLambda: Sets up a Lambda function to perform health check pings to keep the Rails Lambda warm, triggered by a CloudWatch Events rule. 
4. HealthCheckRule: Configures a CloudWatch Events rule to invoke the PingHealthCheckLambda every 3 minutes for health checks.
5. PermissionForEventsToInvokeLambda: Grants CloudWatch Events permission to invoke the PingHealthCheckLambda.
6. JobsLambda: Defines a Lambda function to process jobs from the JobsQueue, with policies for accessing SSM parameters and SQS.
7. RailsLambdaUrlCF: Creates a Lambda function URL for the RailsLambda with no authentication required.
CloudFrontDistribution: Sets up a CloudFront distribution to serve content from the RailsLambda function, using the function URL as the origin domain.
9. AcmCertificate: Requests an ACM certificate for the domain app-${RailsEnv}.example.com, validated via DNS.
10. Route53Record: Creates a Route 53 DNS record to map the domain app-${RailsEnv}.example.com to the CloudFront distribution.

To me this is basic config i want for every app +/- the queue for jobs.

I also use a prewarming function because it seems cheaper than Provisioned Concurrency. According to https://calculator.aws/#/createCalculator/Lambda

Number of requests: 1 per minute * (60 minutes in an hour x 730 hours in a month) = 43800 per month
Amount of memory allocated: 1024 MB x 0.0009765625 GB in a MB = 1 GB
Amount of ephemeral storage allocated: 512 MB x 0.0009765625 GB in a MB = 0.5 GB
Pricing calculations
43,800 requests x 150 ms x 0.001 ms to sec conversion factor = 6,570.00 total compute (seconds)
1 GB x 6,570.00 seconds = 6,570.00 total compute (GB-s)
6,570.00 GB-s - 400000 free tier GB-s = -393,430.00 GB-s
Max (-393430.00 GB-s, 0 ) = 0.00 total billable GB-s
43,800 requests - 1000000 free tier requests = -956,200 monthly billable requests
Max (-956200 monthly billable requests, 0 ) = 0.00 total monthly billable requests
Tiered price for: 0.00 GB-s
Total tier cost = 0.00 USD (monthly compute charges)
0.50 GB - 0.5 GB (no additional charge) = 0.00 GB billable ephemeral storage per function
Lambda costs - With Free Tier (monthly): 0.00 USD

With that being said here is the template.yml and I hope I can save someone searching for cloudfront config a few hours getting the function url

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: App

Parameters:
  SubnetIds:
    Type: CommaDelimitedList
    Description: Comma-separated list of subnet IDs for the Lambda function's VPC configuration.
  SecurityGroupIds:
    Type: CommaDelimitedList
    Description: Comma-separated list of security group IDs for the Lambda function's VPC configuration.
  RailsEnv:
    Type: String
    Default: staging

Globals:
  Function:
    VpcConfig:
      SubnetIds: !Ref SubnetIds
      SecurityGroupIds: !Ref SecurityGroupIds
    Architectures:
      - arm64
    AutoPublishAlias: live
    DeploymentPreference:
      Type: AllAtOnce
    Environment:
      Variables:
        ENV_VAR: !Sub "x-crypteia-ssm-path:/app/${RailsEnv}"
        ENV_VAR2: x-crypteia
        JOBS_QUEUE_NAME: !GetAtt JobsQueue.QueueName
        RAILS_ENV: !Ref RailsEnv
    Timeout: 30

Resources:
  JobsQueue:
    Type: AWS::SQS::Queue
    Properties:
      ReceiveMessageWaitTimeSeconds: 0
      VisibilityTimeout: 300

  RailsLambda:
    Type: AWS::Serverless::Function
    Description: !Sub "Rails application Lambda function for ${RailsEnv} environment"
    Metadata:
      DockerContext: .
      Dockerfile: Dockerfile
      DockerTag: web
    Properties: 
      Policies:
        - Statement:
            - Effect: Allow
              Action: ["ssm:Get*", "ssm:Describe*"]
              Resource:
                - !Sub arn:aws:ssm:*:${AWS::AccountId}:parameter/app/*
        - AWSLambdaVPCAccessExecutionRole
        - Version: "2012-10-17"
          Statement:
            - Effect: Allow
              Action: ["ssm:Get*", "ssm:Describe*"]
              Resource:
                - !Sub arn:aws:ssm:*:${AWS::AccountId}:parameter/app/*
            - Effect: Allow
              Action:
                - sqs:*
              Resource:
                - !Sub arn:aws:sqs:${AWS::Region}:${AWS::AccountId}:${JobsQueue.QueueName}
      FunctionUrlConfig:
        AuthType: NONE
      MemorySize: 1792
      PackageType: Image

  PingHealthCheckLambda:
    Type: AWS::Serverless::Function
    Description: !Sub "Lambda function for health check pings in ${RailsEnv} environment to keep Rails lambda warm"
    Properties:
      Handler: ping.handler
      Runtime: ruby3.2
      CodeUri: src/
      Policies:
        - AWSLambdaBasicExecutionRole
        - Statement:
            Effect: Allow
            Action: "lambda:InvokeFunction"
            Resource: !GetAtt RailsLambda.Arn

  HealthCheckRule:
    Type: AWS::Events::Rule
    Properties:
      ScheduleExpression: "cron(0/3 * * * ? *)"  # Every 3 minutes
      Targets:
        - Id: "HealthCheckLambdaTarget"
          Arn: !GetAtt PingHealthCheckLambda.Arn
          Input: '{"EVENT_RULE": "HEALTHCHECK"}'

  PermissionForEventsToInvokeLambda: 
    Type: AWS::Lambda::Permission
    Properties: 
      FunctionName: 
        Ref: "PingHealthCheckLambda"
      Action: "lambda:InvokeFunction"
      Principal: "events.amazonaws.com"
      SourceArn: 
        Fn::GetAtt: 
          - "HealthCheckRule"
          - "Arn"

  JobsLambda:
    Type: AWS::Serverless::Function
    Description: !Sub "Lambda function for processing jobs from SQS in ${RailsEnv} environment"
    Metadata:
      DockerContext: .
      Dockerfile: Dockerfile
      DockerTag: jobs
    Properties:
      Events:
        SQSJobs:
          Type: SQS
          Properties:
            Queue: !GetAtt JobsQueue.Arn
            BatchSize: 1
            FunctionResponseTypes:
              - ReportBatchItemFailures
      ImageConfig:
        Command: ["config/environment.Lambdakiq.cmd"]
      MemorySize: 1792
      PackageType: Image 
      Policies:
        - Version: "2012-10-17"
          Statement:
            - Effect: Allow
              Action: ["ssm:Get*", "ssm:Describe*"]
              Resource:
                - !Sub arn:aws:ssm:*:${AWS::AccountId}:parameter/app/*  
            - Effect: Allow
              Action:
                - sqs:*
              Resource:
                - !Sub arn:aws:sqs:${AWS::Region}:${AWS::AccountId}:${JobsQueue.QueueName}
      Timeout: 300

  CloudFrontDistribution:
    Type: AWS::CloudFront::Distribution
    DependsOn: RailsLambda
    Properties:
      DistributionConfig:
        Aliases:
          - !Sub "app-${RailsEnv}.example.com"
        Comment: !Sub "app-${RailsEnv} lamby managed"
        Enabled: true
        HttpVersion: "http2"
        IPV6Enabled: true
        PriceClass: "PriceClass_100"
        DefaultCacheBehavior:
          AllowedMethods:
            - DELETE
            - GET
            - HEAD
            - OPTIONS
            - PATCH
            - POST
            - PUT
          CachePolicyId: "658327ea-f89d-4fab-a63d-7e88639e58f6"
          CachedMethods:
            - GET
            - HEAD
          Compress: false
          DefaultTTL: 0
          MaxTTL: 0
          MinTTL: 0
          OriginRequestPolicyId: "b689b0a8-53d0-40ab-baf2-68738e2966ac"
          ResponseHeadersPolicyId: "5cc3b908-e619-4b99-88e5-2cf7f45965bd"
          TargetOriginId: "DefaultOrigin"
          ViewerProtocolPolicy: "redirect-to-https"
        Origins:
          - Id: "DefaultOrigin"
            DomainName: 
              # Remove 'https://' from the FunctionUrl
              !Join
                - ''  # Join with an empty delimiter
                - 
                  # Select the domain part of the URL
                  - !Select 
                      - 2  # Select the third element (index 2) from the split parts
                      - !Split 
                          - '/'  # Split the URL by '/'
                          - !GetAtt RailsLambdaUrl.FunctionUrl  # Get the full Function URL
            CustomOriginConfig:
              HTTPPort: 80
              HTTPSPort: 443
              OriginKeepaliveTimeout: 5
              OriginProtocolPolicy: "https-only"
              OriginReadTimeout: 30
              OriginSSLProtocols:
                - "SSLv3"
                - "TLSv1"
            OriginCustomHeaders:
              - HeaderName: "X-Forwarded-Host"
                HeaderValue: !Sub "app-${RailsEnv}.example.com"
        ViewerCertificate:
          AcmCertificateArn: !Ref AcmCertificate
          SslSupportMethod: "sni-only"
          MinimumProtocolVersion: "TLSv1.2_2021"
        Restrictions:
          GeoRestriction:
            RestrictionType: "none"

  AcmCertificate:
    Type: AWS::CertificateManager::Certificate
    Properties:
      DomainName: !Sub "app-${RailsEnv}.example.com"
      ValidationMethod: DNS
      DomainValidationOptions:
        - DomainName: !Sub "app-${RailsEnv}.example.com"
          HostedZoneId: "YOUR_ZONE_ID"

  Route53Record:
    Type: AWS::Route53::RecordSet
    Properties:
      Name: !Sub "app-${RailsEnv}.example.com"
      Type: "A"
      HostedZoneId: "YOUR_ZONE_ID"
      AliasTarget:
        DNSName: !GetAtt CloudFrontDistribution.DomainName
        HostedZoneId: "YOUR_ZONE_ID"
Outputs:
  RailsLambdaUrl:
    Description: Lambda Function URL
    Value: !GetAtt RailsLambdaUrl.FunctionUrl
jeremiahlukus commented 5 months ago

Also to enable lambda insights locally i ran

export URL=$(aws lambda get-layer-version-by-arn --arn arn:aws:lambda:us-east-1:580247275435:layer:LambdaInsightsExtension-Arm64:19 --query Content.Location --output text --region us-east-1)

curl -o LambdaInsightsExtension-Arm64.zip $URL

Then in my Dockerfile after coping the code over

COPY LambdaInsightsExtension-Arm64.zip /opt
RUN unzip /opt/LambdaInsightsExtension-Arm64.zip -d /opt/

Then in your lambda add

        - CloudWatchLambdaInsightsExecutionRolePolicy

The way to do this in the aws docs does not work for arm64

jeremiahlukus commented 5 months ago

Lastly since i wanted to use arm64 I was unable to deploy using github actions. I am using code build instead

data "local_file" "buildspec_local" {
    filename = "${path.module}/buildspec.yml"
}

resource "aws_codebuild_project" "name" {
  badge_enabled          = false
  build_timeout          = 60
  concurrent_build_limit = 5
  description            = null
  encryption_key         = "arn:aws:kms:us-east-1:533085732793:alias/aws/s3"
  name                   = "APPNAME-remote"
  project_visibility     = "PRIVATE"
  queued_timeout         = 480
  resource_access_role   = null
  service_role           = "arn:aws:iam::533085732793:role/code-build"
  source_version         = "main"
  tags                   = {}
  tags_all               = {}
  artifacts {
    artifact_identifier    = null
    bucket_owner_access    = null
    encryption_disabled    = false
    location               = null
    name                   = null
    namespace_type         = null
    override_artifact_name = false
    packaging              = null
    path                   = null
    type                   = "NO_ARTIFACTS"
  }
  cache {
    location = null
    modes    = ["LOCAL_DOCKER_LAYER_CACHE", "LOCAL_CUSTOM_CACHE"]
    type     = "LOCAL"
  }
  environment {
    certificate                 = null
    compute_type                = "BUILD_GENERAL1_SMALL"
    image                       = "aws/codebuild/amazonlinux2-aarch64-standard:3.0"
    image_pull_credentials_type = "CODEBUILD"
    privileged_mode             = true
    type                        = "ARM_CONTAINER"
  }
  logs_config {
    cloudwatch_logs {
      group_name  = null
      status      = "ENABLED"
      stream_name = null
    }
    s3_logs {
      bucket_owner_access = null
      encryption_disabled = false
      location            = null
      status              = "DISABLED"
    }
  }

  source {
    type            = "GITHUB"
    location        = "https://github.com/USERNAME/REPO.git"
    git_clone_depth = 1
    buildspec       = data.local_file.buildspec_local.content
    git_submodules_config {
      fetch_submodules = true
    }
  }
}

buildspec is simple and it shouldn't get ran because im overriding it in my github action

version: 0.2

phases:
  install:
    runtime-versions:
      ruby: 3.2
  build:
    commands:
      - echo "== DEPLOY =="
      - RAILS_ENV=review bin/deploy

your ruby version needs to be 3.2.2 in order to use the AWS runtime.

Now i created a github workflow so i dont have to go to codebuild and see the progress of my build

      - name: Run CodeBuild
      uses: aws-actions/aws-codebuild-run-build@v1
      with:
        project-name: ${{ inputs.PROJECT_NAME }}
        disable-source-override: false
        source-version-override: ${{ github.sha }}
        buildspec-override:   |
          version: 0.2
          phases:
            install:
              runtime-versions:
                ruby: 3.2
            build:
              commands:
                - echo "== DOCKER LOGIN =="
                - docker login -u ${{ inputs.DOCKER_USER }} -p ${{ inputs.DOCKER_PASS }}
                - echo "== DEPLOY =="
                - cd rails_api
                - RAILS_ENV=${{ inputs.ENVIRONMENT}} bin/deploy

Im using the same code build project to deploy all envs i just pass in the rails env and it deploys that env.