[lambda] specify multiple folders as a single lambda asset

efimk-lu commented 6 years ago

Hello, Great framework, keep up the good work.

Right now in order to define a lambda function you need to set where the code resides, e.g. code: lambda.Code.asset('lambda'). Sometimes you would like to define multiple folders, for example the actual code and node_modules, although packing can be done as a preprocess step before actually running cdk deploy, it would be cleaner to do it as part of the lambda configuration.

rix0rrr commented 6 years ago

Wouldn't you pick the single directory that contains both your code and the node_modules directory, like this?

.
|-- my_code.ts
`-- node_modules
    `-- some_library

efimk-lu commented 6 years ago

I've been thinking on the following structure

/ ... package.json ... Various conf file ... src ...... my_code.js ... test ...... test_my_code.js ... node_modules ... cdk ...... deploy.js

eladb commented 6 years ago

I think that's definitely an interesting idea. One would also need to map the sources to their location in the destination tree (for example, you would probably want src/my_code.js in the root of the destination tree and node_modules next to it, right?

efimk-lu commented 6 years ago

@eladb right

justinmchase commented 5 years ago

I have a mono-repo and so I can't just include the root of the project. I use tsc to compile code from src/ -> dist/, from ts to js. My node_modules exists at the root of the project and I don't want to include all dependencies even.

Something like this would be great:

    const example = new Function(this, 'Example'), {
      runtime: Runtime.NodeJS810,
      handler: 'index.handler',
      code: Code.files([
        'dist/funcs/example/*',
        'node_modules/*'
        '!node_modules/aws-sdk'
    });

Similar to the way gulp would let you include src files.

Right now I'm looking at adding a package.json into my func folder and running npm i --production but its a bit of a pain, I don't have any mechanism for copying non-code files currently.

eladb commented 5 years ago

I can see the value in this, and I think we can support something like that, but I was wondering if this is actually the best approach for bundling node.js dependencies for Lambda. Excluding certain modules from the root of your node_modules directory might really not be the right approach. I'd argue that your npm i --production approach to generate a clean node_modules tree that includes only the deps needed for your runtime code is a much more robust approach.

I wonder if perhaps there's something we can add to the CDK such that utilizing this approach will be straightforward (i.e. run npm install from within the toolkit?)

It's definitely a topic of interest for us, which we would love to address end-to-end (for node and all other languages as well). Copy: @sam-goodwin

justinmchase commented 5 years ago

@eladb I agree after digging into it more.

I wrote a little script to do this, to illustrate the issue:

#!/bin/bash
set -e

function prep-func {
  func=$1

  echo "$func:"
  cp src/funcs/$func/package.json dist/funcs/$func/package.json
  cp src/funcs/$func/package-lock.json dist/funcs/$func/package-lock.json
  pushd dist/funcs/$func      &> /dev/null
  npm i --production --silent
  popd                        &> /dev/null
}

echo "compiling..." 
npx tsc

echo "preparing functions..."
prep-func "shared"
prep-func "exportDelivery"

echo "deploying to amazon..."
npx cdk deploy --app dist/cloud/index.js

echo "done."

I think its fine to let people resolve this on their own on second thought, because I'm not sure how you could really solve this in a general way. These are steps for the pre-deploy build system of each individual project, and if I have a mono-repo I need to ensure that each of those sup-repos are building their dependencies at the correct time with the correct settings, not you.

nija-at commented 5 years ago

still relevant

shahbour commented 4 years ago

Any plan on this feature ?

eladb commented 4 years ago

Perhaps @jogold will be interested to add this capability to our new shiny staging system.

gabrielpeery commented 3 years ago

Hey has there been any movement on this since August? I could really use this feature in my mono-repo.

zarnoevic commented 3 years ago

+1

bes commented 3 years ago

Would love this feature, have a somewhat intricate folder structure and I really don't want to duplicate a lot of files...

ronnathaniel commented 3 years ago

+1

github-actions[bot] commented 2 years ago

This issue has received a significant amount of attention so we are automatically upgrading its priority. A member of the community will see the re-prioritization and provide an update on the issue.