search

FieryCod / holy-lambda

The extraordinary simple, performant, and extensible custom AWS Lambda runtime for Clojure.
https://fierycod.github.io/holy-lambda
MIT License
340 stars 20 forks source link

[BUG] bb stack:invoke fails with babashka library error, event if runtime is java #58

Closed mcuervoe closed 3 years ago

mcuervoe commented 3 years ago

Describe the bug A correct code, one that can be deployed in AWS with the native or java runtime, fails when called with bb stack:invoke using java runtime. The lambda function uses the cognitect/aws-lib library

NOTE: I can not guarantee is correct with babashka runtime because the layers + application is bigger than 250Mb limit imposed by AWS, so the code can't be deployed in AWS.

To Reproduce

bb stack:invoke :runtime java :event-file resources/event-test-schedule.json :envs-file envs.json

This produces this output:

bb stack:invoke :runtime :java :event-file resources/event-test-schedule.json :envs-file envs.json 
[holy-lambda] Command <stack:invoke>
[holy-lambda] Build is stale. Consider recompilation via stack:compile
Invoking com.nubizzi.normon-monitor.core.ServerMonitor (java8)
Decompressing /home/miguel/git/onetech/normon-monitor/.holy-lambda/build/output.jar
arn:aws:lambda:eu-west-3:395496933615:layer:holy-lambda-babashka-runtime:2 is already cached. Skipping download
Skip pulling image and use local one: samcli/lambda:java8-2ff027c923cd792b48aa34530.

Mounting /tmp/tmpew282zgw as /var/task:ro,delegated inside runtime container
START RequestId: e9fcea64-8937-404a-87d7-a9b2f8e512ba Version: $LATEST
Downloading pod org.babashka/aws (0.0.5)
Successfully installed pod org.babashka/aws (0.0.5)
----- Error --------------------------------------------------------------------
Type:     clojure.lang.ExceptionInfo
Message:  Could not resolve symbol: clojure.lang.LockingTransaction/isRunning
Location: /var/task/clojure/tools/logging.clj:59:15
Phase:    analysis

----- Context ------------------------------------------------------------------
55:   [logger level throwable message]
56:   (if (case *force*
57:         :agent  true
58:         :direct false
59:         (and (clojure.lang.LockingTransaction/isRunning)
                  ^--- Could not resolve symbol: clojure.lang.LockingTransaction/isRunning
60:              (*tx-agent-levels* level)))
61:     (send-off *logging-agent*
62:       (fn [_#] (impl/write! logger level throwable message)))
63:     (impl/write! logger level throwable message))
64:   nil)

time="2021-06-22T14:36:32.503" level=error msg="Init failed" InvokeID= error="Runtime exited with error: exit status 1"
time="2021-06-22T14:36:32.503" level=error msg="INIT DONE failed: Runtime.ExitError"

Expected behavior The same behavior that I have with native runtime

bb stack:invoke :runtime native :event-file resources/event-test-schedule.json :envs-file envs.json
Invoking com.nubizzi.normon-monitor.core.ServerMonitor (provided)
Decompressing /home/miguel/git/onetech/normon-monitor/.holy-lambda/build/latest.zip
arn:aws:lambda:eu-west-3:395496933615:layer:holy-lambda-babashka-runtime:2 is already cached. Skipping download
Skip pulling image and use local one: samcli/lambda:provided-2ff027c923cd792b48aa34530.

Mounting /tmp/tmpu5ggg5n2 as /var/task:ro,delegated inside runtime container
START RequestId: 4a6c256a-db61-4701-8088-fffa22dd58b9 Version: $LATEST
....
END RequestId: 4a6c256a-db61-4701-8088-fffa22dd58b9
REPORT RequestId: 4a6c256a-db61-4701-8088-fffa22dd58b9  Init Duration: 1.00 ms  Duration: 5190.54 ms    Billed Duration: 5200 ms        Memory Size: 256 MB     Max Memory Used: 256 MB

Extra information:

[holy-lambda] --------------------------------------- [holy-lambda] Checking health of holy-lambda stack [holy-lambda] Home directory is: /home/miguel [holy-lambda] Project directory is: /home/miguel/git/onetech/normon-monitor [holy-lambda] AWS SAM version: SAM CLI, version 1.23.0 [holy-lambda] AWS CLI version: aws-cli/2.0.42 Python/3.7.3 Linux/4.19.128-microsoft-standard exe/x86_64.ubuntu.18 [holy-lambda] AWS directory is: /home/miguel/.aws [holy-lambda] AWS directory exists?: true [holy-lambda] Docker version: Docker version 20.10.6, build 370c289 [holy-lambda] Babashka tasks sha: 50c055b1b9addf6bfc25bca8a1deecfd3af5c36f [holy-lambda] Babashka tasks version: 0.1.49 [holy-lambda] Babashka version: babashka v0.4.4 [holy-lambda] Runtime: :native [holy-lambda] Runtime entrypoint: com.nubizzi.normon-monitor.core [holy-lambda] Stack name: normon-monitor-96661897a56449f1954de8a4627c40d1-stack [holy-lambda] S3 Bucket name: normon-monitor-96661897a56449f1954de8a4627c [holy-lambda] S3 Bucket prefix: holy-lambda [holy-lambda] S3 Bucket exists?: true [holy-lambda] ---------------------------------------

[holy-lambda] :runtime looks good [holy-lambda] :runtime:entrypoint looks good [holy-lambda] :stack:capabilities looks good [holy-lambda] :runtime:bootstrap-file does not exists. Default bootstrap file for :native runtime will be used! [holy-lambda] :runtime:version is supported only for :babashka runtime [holy-lambda] :runtime:pods are supported only for :babashka runtime [holy-lambda] :stack:name looks good [holy-lambda] property :mvn/local-repo in file deps.edn is correct [holy-lambda] property :mvn/local-repo in file bb.edn is correct [holy-lambda] Syncing stack is not required [holy-lambda] :infra:region definition looks good [holy-lambda] :infra:bucket-prefix looks good [holy-lambda] :stack:template looks good [holy-lambda] :infra:bucket-name looks good [holy-lambda] Required commands ["aws" "sam" "bb" "docker" "clojure" "zip" "id" "clj-kondo" "bash"] installed!

[holy-lambda] Validating template.yml 2021-06-22 16:47:51 Loading policies from IAM... 2021-06-22 16:47:55 Finished loading policies from IAM. /home/miguel/git/onetech/normon-monitor/template.yml is a valid SAM Template

 - deps.edn:

````clojure
{:deps           {org.clojure/clojure            {:mvn/version "1.10.3"}
                  org.clojure/spec.alpha         {:mvn/version "0.2.194"}
                  org.clojure/tools.logging      {:mvn/version "1.1.0"}
                  io.github.FieryCod/holy-lambda {:mvn/version "0.2.1"}
                  clj-commons/clj-ssh            {:mvn/version "0.5.15"}
                  http-kit/http-kit              {:mvn/version "2.5.3"}
                  org.clojure/data.json          {:mvn/version "2.3.1"}
                  mount/mount                    {:mvn/version "0.1.16"}
                  com.cognitect.aws/api          {:mvn/version "0.8.505"},
                  com.cognitect.aws/endpoints    {:mvn/version "1.1.11.1001"}
                  com.cognitect.aws/ec2          {:mvn/version "811.2.889.0", :aws/serviceFullName "Amazon Elastic Compute Cloud"}
                  com.cognitect.aws/s3           {:mvn/version "811.2.889.0"}}

 ;; Local repositories should be downloaded to `.holy-lambda` directory
 ;; This way project artifacts are separated from global ~/.m2 and only project artifacts
 ;; are deployed to AWS Lambda
 :mvn/local-repo ".holy-lambda/.m2"
 :paths          ["src" "resources"]
 :aliases        {:dev {:extra-paths ["src/dev"]
                        :extra-deps  {org.clojure/tools.namespace {:mvn/version "1.1.0"}
                                      expound/expound             {:mvn/version "0.8.7"}}}}}
FieryCod commented 3 years ago

Are you having babashka layer in template.yml?

EDIT: It seems to me that you are mixing native/java runtime with babashka layer.

Regarding babashka limit please create a separate issue for it.

mcuervoe commented 3 years ago

This is the template.yml file that I am using

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  Example basic lambda using `holy-lambda` micro library

Parameters:
  Runtime:
    Type: String
    Default: <HOLY_LAMBDA_RUNTIME>
  Timeout:
    Type: Number
    Default: 30
  MemorySize:
    Type: Number
    Default: 256
  CodeUri:
    Type: String
    Default: <HOLY_LAMBDA_CODE_URI>
  Entrypoint:
    Type: String
    Default: <HOLY_LAMBDA_ENTRYPOINT>

Globals:
  Function:
    Runtime: !Ref Runtime
    Timeout: !Ref Timeout
    MemorySize: !Ref MemorySize
    CodeUri: !Ref CodeUri
    Environment:
      Variables:
        Entrypoint: !Ref Entrypoint

Resources:
  LambdaExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - lambda.amazonaws.com
            Action:
              - sts:AssumeRole
      Path: "/"
      Policies:
        - PolicyName: root
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Effect: Allow
                Action:
                  - logs:*
                Resource: arn:aws:logs:*:*:*
              - Effect: Allow
                Action:
                  - ec2:DescribeInstances
                  - ec2:RebootInstances
                  - ec2:DescribeNetworkInterfaces
                  - ec2:CreateNetworkInterface
                  - ec2:DeleteNetworkInterface
                  - ec2:DescribeInstances
                  - ec2:AttachNetworkInterface
                Resource: "*"
              - Effect: Allow
                Action:
                  - s3:GetObject
                Resource: arn:aws:s3:::123456789-config/*

  ServerMonitorFunctionDev:
    Type: AWS::Serverless::Function
    Properties:
      FunctionName: ServerMonitorDev
      Handler: com.nubizzi.normon-monitor.core.ServerMonitor
      Layers:
        - arn:aws:lambda:eu-west-3:395496933615:layer:holy-lambda-babashka-runtime:2
      Environment:
        Variables:
          S3_CONFIG_BUCKET: 123456789-config
          S3_CONFIG_KEY: servers-dev.edn
      Role: !GetAtt
        - LambdaExecutionRole
        - Arn
      Events:
        RebootEvent:
          Type: SNS
          Properties:
            Topic: !Ref SNSTopicAlarmDev
        PeriodicEvent:
          Type: Schedule
          Properties:
            Schedule: 'rate(1 minute)'
            Name: MonitorNormonDev
            Description: Schedule to monitor apps in dev
            Enabled: true
      EventInvokeConfig:
        MaximumRetryAttempts: 0
      VpcConfig:
        SecurityGroupIds:
          - sg-abcdec
        SubnetIds:
          - subnet-777222111

  SNSTopicAlarmDev:
    Type: AWS::SNS::Topic
    Properties:
      DisplayName: Alarm Dev
      Tags:
        - Key: Environment
          Value: DEV
      TopicName: AlarmDev
FieryCod commented 3 years ago

@mcuervoe You cannot use:

  Layers:
        - arn:aws:lambda:eu-west-3:395496933615:layer:holy-lambda-babashka-runtime:2

with java/native runtime.

mcuervoe commented 3 years ago

Thank Karol! That solves the problem.

The development cycle is much faster now when I do not have to wait for the compilation of the native image to test the lambda in my local environment

FieryCod commented 3 years ago

You welcome @mcuervoe :) I will close this issue since it's not the issue with HL :)

If you would like to use babashka runtime you would have to use https://github.com/tzzh/pod-tzzh-aws instead of aws-api pod.