little-bear-labs / aws-utils

Apache License 2.0
56 stars 35 forks source link

Lambda datasource docker #152

Open Frikitrok opened 5 years ago

Frikitrok commented 5 years ago

As i see from code lambda runner just execute 'sls invoke local'. My solution were build expecting lambda execution in docker, so there is no way it will work with simulator. Is it possible to add --docker arg somehow?

Also quick question: after i sent request to local graphql, it show (see below) and stuck, after timeout my request fails with "Remote end closed connection without response". Any suggestion whats going on?

Debugger listening on ws://127.0.0.1:9229/87bc4755-e73c-419e-9298-bd9c118ae8de For help, see: https://nodejs.org/en/docs/inspector

cbaron commented 5 years ago

@Frikitrok -- could you describe your issues in more detail? Seems like there are two:

1) You want the lambda to be executed within docker. I believe that's already happening. The emulator runs sls invoke local in a child process of the emulator. What would the --docker arg do?

2) Remote end closed connection without response -- you might need to open ports on your docker container. I can help debug if you can provide relevant pieces of source code.

praisegeek commented 5 years ago

@cbaron I am doing this without docker. The LambdaRunner spins up inspection which timeouts after 30secs with the below response

Assertion failed: (!uv__is_closing(handle)), function uv_close, file ../deps/uv/src/unix/core.c, line 117.

The Invocation appears not to hit the Lambda function. I had tried both ways, using buildPrefix alongside serverless-webpack and without.

In a nutshell, Invoking Lambda functions from resolver templates fail. Only tested with default runtime, Node ">= 6 < 10.x".

cbaron commented 5 years ago

@praisegeek -- can you paste some source code? Its tough to debug when all I see is the error.

Also, is this a new project for you? If so, I would suggest migrating to AWS's Amplify project which now integrates thisemulator: https://aws.amazon.com/blogs/aws/new-local-mocking-and-testing-with-the-amplify-cli/

praisegeek commented 5 years ago

Hi @cbaron, thanks. You can find the code below.

serverless.yml

service: testapp

frameworkVersion: ">=1.48.1 <2.0.0"

plugins:
  - serverless-dotenv-plugin
  - serverless-stage-manager
  - serverless-webpack
  - serverless-dynamodb-local
  - serverless-appsync-plugin
  - serverless-appsync-offline
  - serverless-offline

package:
  exclude:
    - node_modules/**

provider:
  name: aws
  runtime: nodejs8.10
  profile: test
  stage: ${{opt:stage, 'local'}}
  region: eu-west-2
  variableSyntax: "\\${{([ ~:a-zA-Z0-9._\\'\",\\-\\/\\(\\)]+?)}}"
  environment:
    AWS_STORE_BUCKET: "${{self:service}}-store"

custom:
  accountId: ${AWS::AccountId}

  appSync:
    name: ${{self:provider.stage}}-${{self:service}}-api
    authenticationType: API_KEY
    userPoolConfig:
      awsRegion: ${{self:provider.region}}
      defaultAction: ALLOW
      userPoolId: !Ref CognitoUserPoolMyUserPool
    region: ${{self:provider.region}}
    serviceRole: "AppSyncServiceRole"
    schema: schema.graphql
    mappingTemplates:
      - ${{file(includes/mappingtemplates.yml)}}
    dataSources:
      - type: AWS_LAMBDA
        name: Lambda
        description: "Lambda DataSource"
        config:
          lambdaFunctionArn: { Fn::GetAtt: [GraphqlLambdaFunction, Arn] }
          serviceRoleArn: { Fn::GetAtt: [AppSyncLambdaServiceRole, Arn] }
          functionName: graphql

  serverless-offline:
     port: 4000

  appsync-offline:
    port: 62222
    region: ${{self:provider.region}}

  appsync-emulator:
    buildPrefix: .webpack/service

  dynamodb:
    stages:
      - dev
      - local
      - prod

   client:
      endpoint: "http://localhost:8000"
      region: "localhost"
      accessKeyId: a
      secretAccessKey: a

  webpack:
    packager: "yarn"
    webpackConfig: ./webpack.config.js
    includeModules:
      forceInclude:
        # - pg
      forceExculde:
        - aws-sdk

functions:
  graphql:
    handler: handlers/graphql.handler

resources:
  - ${{file(roles/appsync-roles.yml)}}

schema.graphql


type Tenant {
  id: ID!
  name: String!
  website: String 
  phone: String!
  createdAt: String
}

input CreateTenantInput {
  name: String!
  website: String
  phone: String!
}

type Mutation {
  createTenant(input: CreateTenantInput!) : Tenant
}

schema {
  mutation: Mutation
}

mappingtemplates.yml

- dataSource: Lambda
  type: Mutation
  field: createTenant
  request: "tenants/Mutation.createTenant.request.vtl"
  response: "tenants/Query.listTenants.response.vtl"

Mutation.createTenant.request.vtl

{
  "version": "2017-02-28",
  "operation": "Invoke",
  "payload": {
    "field": "createTenant",
    "arguments": $util.dynamodb.toMapValuesJson($ctx.args.input)
  }
}

handlers/graphql.js

module.exports.handler = (event, context, callback) => {

  console.log("Received event {}", JSON.stringify(event, 3));

  switch(event.field) {
    case "createTenant":
      callback(null, { test: "ok" } )

    default:
      callback(null, { test: "default" } )
  }
}

webpack.config.js

const path = require("path");

// eslint-disable-next-line import/no-extraneous-dependencies
const nodeExternals = require("webpack-node-externals");
// eslint-disable-next-line import/no-extraneous-dependencies
const slsw = require("serverless-webpack");

const CopyWebpackPlugin = require("copy-webpack-plugin");

module.exports = {
  entry: slsw.lib.entries,
  mode: slsw.lib.webpack.isLocal ? "development" : "production",
  target: "node",
  node: { __dirname: false },
  externals: [nodeExternals()],
  plugins: [
    new CopyWebpackPlugin([
      {
        from: "email-templates",
        to: "email-templates",
        toType: "dir"
      }
    ])
  ],
  module: {
    rules: [
      {
        test: /\.js$/,
        use: [
          {
            loader: "babel-loader",
            options: {
              presets: [["env", { targets: { node: "8.10" } }]]
            }
          }
        ]
      }
    ]
  },
  output: {
    libraryTarget: "commonjs",
    path: path.join(__dirname, ".webpack"),
    filename: "[name].js"
  }
};

package.json

{
  "name": "backend",
  "version": "1.0.0",
  "main": "index.js",
  "license": "MIT",
  "devDependencies": {
    "@babel/core": "^7.4.5",
    "babel-core": "^6.26.3",
    "babel-loader": "^8.0.6",
    "babel-preset-env": "^1.7.0",
    "babel-preset-jest": "^24.6.0",
    "copy-webpack-plugin": "^5.0.3",
    "faker": "^4.1.0",
    "ignore-loader": "^0.1.2",
    "json": "^9.0.6",
    "jsonfile": "^5.0.0",
    "raw-loader": "^3.0.0",
    "serverless-appsync-offline": "praisegeek/serverless-appsync-offline",
    "serverless-appsync-plugin": "^1.1.2",
    "serverless-dotenv-plugin": "^2.1.1",
    "serverless-dynamodb-local": "^0.2.38",
    "serverless-offline": "^5.5.0",
    "serverless-stage-manager": "^1.0.5",
    "serverless-webpack": "^5.3.1",
    "ts-loader": "^6.0.4",
    "webpack": "^4.35.0",
    "webpack-bundle-analyzer": "^3.3.2",
    "webpack-node-externals": "^1.7.2"
  },
  "scripts": {
    "start": "sls offline start",
  },
  "dependencies": {
    "@babel/plugin-transform-runtime": "^7.5.5",
    "@babel/preset-env": "^7.5.5",
    "@babel/runtime": "^7.5.5",
    "async": "^3.1.0",
    "aws-sdk": "^2.484.0",
    "babel-polyfill": "^6.26.0",
    "mjml": "^4.4.0-beta.2",
    "mjml-utils": "^2.2.1",
    "node-fetch": "^2.6.0",
    "nodemailer": "^6.2.1"
  }
}
cbaron commented 5 years ago

@praisegeek -- is

webpack/service

the folder where your handler is being compiled to?

praisegeek commented 5 years ago

@cbaron Yeah. I can also find the files in the said directory.

.webpack/service/handlers/graphql.js
praisegeek commented 5 years ago

Same effect even without using webpack.

cbaron commented 5 years ago

Did you check this thread? https://github.com/serverless-heaven/serverless-webpack/issues/492

cbaron commented 5 years ago

I'm not sure if this is related to the emulator.

praisegeek commented 5 years ago

@cbaron I will check that thread out and revert back. It might be the root cause.

praisegeek commented 5 years ago

Problem still persists.

praisegeek commented 5 years ago

@cbaron I just made a pull request which fixes this. #163