mountCode: True with serverless-python-requirements

[maranqz]

If mountCode: True deploy crashing other hand mountCode: False deploy is ok. serverless-python-requirements serverless-localstack

serverless.yml

Slightly modified https://github.com/serverless/examples/tree/master/aws-python-line-echo-bot ```yaml # ⚠️⚠️ REPLACE THIS COMMENT WITH FULL serverless.yml CONTENT service: aws-python-line-echo-bot provider: name: aws runtime: python3.7 functions: line_bot: handler: handler.webhook events: - http: path: /webhook method: POST plugins: - serverless-localstack - serverless-python-requirements custom: localstack: debug: true stages: - local host: http://localhost # optional - LocalStack host to connect to autostart: true # optional - start LocalStack in Docker on Serverless deploy endpoints: # This section is optional - can be used for customizing the target endpoints S3: http://localhost:4572 Lambda: http://localhost:4574 lambda: # Enable this flag to improve performance mountCode: True docker: # Enable this flag to run "docker ..." commands as sudo sudo: False stages: local: ```

serverless deploy --stage local output

``` aws-python-line-echo-bot>sls deploy --stage local Serverless: Load command interactiveCli Serverless: Load command config Serverless: Load command config:credentials Serverless: Load command create Serverless: Load command install Serverless: Load command package Serverless: Load command deploy Serverless: Load command deploy:function Serverless: Load command deploy:list Serverless: Load command deploy:list:functions Serverless: Load command invoke Serverless: Load command invoke:local Serverless: Load command info Serverless: Load command logs Serverless: Load command metrics Serverless: Load command print Serverless: Load command remove Serverless: Load command rollback Serverless: Load command rollback:function Serverless: Load command slstats Serverless: Load command plugin Serverless: Load command plugin Serverless: Load command plugin:install Serverless: Load command plugin Serverless: Load command plugin:uninstall Serverless: Load command plugin Serverless: Load command plugin:list Serverless: Load command plugin Serverless: Load command plugin:search Serverless: Load command config Serverless: Load command config:credentials Serverless: Load command rollback Serverless: Load command rollback:function Serverless: Load command upgrade Serverless: Load command uninstall Serverless: config.options_stage: local Serverless: serverless.service.custom.stage: undefined Serverless: serverless.service.provider.stage: dev Serverless: config.stage: local Serverless: Using serverless-localstack Serverless: Reconfiguring service apigateway to use http://localhost:4567 Serverless: Reconfiguring service cloudformation to use http://localhost:4581 Serverless: Reconfiguring service cloudwatch to use http://localhost:4582 Serverless: Reconfiguring service lambda to use http://localhost:4574 Serverless: Reconfiguring service dynamodb to use http://localhost:4569 Serverless: Reconfiguring service kinesis to use http://localhost:4568 Serverless: Reconfiguring service route53 to use http://localhost:4580 Serverless: Reconfiguring service firehose to use http://localhost:4573 Serverless: Reconfiguring service stepfunctions to use http://localhost:4585 Serverless: Reconfiguring service es to use http://localhost:4578 Serverless: Reconfiguring service s3 to use http://localhost:4572 Serverless: Reconfiguring service ses to use http://localhost:4579 Serverless: Reconfiguring service sns to use http://localhost:4575 Serverless: Reconfiguring service sqs to use http://localhost:4576 Serverless: Reconfiguring service sts to use http://localhost:4592 Serverless: Reconfiguring service iam to use http://localhost:4593 Serverless: Reconfiguring service ssm to use http://localhost:4583 Serverless: Reconfiguring service rds to use http://localhost:4594 Serverless: Reconfiguring service ec2 to use http://localhost:4597 Serverless: Reconfiguring service elasticache to use http://localhost:4598 Serverless: Reconfiguring service kms to use http://localhost:4599 Serverless: Reconfiguring service secretsmanager to use http://localhost:4584 Serverless: Reconfiguring service logs to use http://localhost:4586 Serverless: Reconfiguring service cloudwatchlogs to use http://localhost:4586 Serverless: Reconfiguring service iot to use http://localhost:4589 Serverless: Reconfiguring service cognito-idp to use http://localhost:4590 Serverless: Reconfiguring service cognito-identity to use http://localhost:4591 Serverless: Reconfiguring service ecs to use http://localhost:4601 Serverless: Reconfiguring service eks to use http://localhost:4602 Serverless: Reconfiguring service xray to use http://localhost:4603 Serverless: Reconfiguring service appsync to use http://localhost:4605 Serverless: Reconfiguring service cloudfront to use http://localhost:4606 Serverless: Reconfiguring service athena to use http://localhost:4607 Serverless: Reconfiguring service S3 to use http://localhost:4572 Serverless: Reconfiguring service Lambda to use http://localhost:4574 Serverless: Warning: Unable to find plugin named: TypeScriptPlugin Serverless: Load command deploy Serverless: Load command requirements Serverless: Load command requirements:clean Serverless: Load command requirements:install Serverless: Load command requirements:cleanCache Serverless: Load command login Serverless: Load command logout Serverless: Load command generate-event Serverless: Load command test Serverless: Load command dashboard Serverless: Load command output Serverless: Load command output:get Serverless: Load command output:list Serverless: Load command param Serverless: Load command param:get Serverless: Load command param:list Serverless: Load command studio Serverless: Load command dev Serverless: Invoke deploy Serverless: Invoke package Serverless: Invoke aws:common:validate Serverless: config.options_stage: local Serverless: serverless.service.custom.stage: undefined Serverless: serverless.service.provider.stage: dev Serverless: config.stage: local Serverless: config.options_stage: local Serverless: serverless.service.custom.stage: undefined Serverless: serverless.service.provider.stage: dev Serverless: config.stage: local Serverless: Invoke aws:common:cleanupTempDir Serverless: Generated requirements from G:\projects\parse\trash\examples\aws-python-line-echo-bot\requirements.txt in G:\projects\parse\trash\examples\aws-python-line-echo-bot\.ser verless\requirements.txt... Serverless: Installing requirements from G:\projects\parse\trash\examples\aws-python-line-echo-bot\.serverless\requirements\requirements.txt ... Serverless: Running ... Serverless: Skip plugin function Package.packageService (lambda.mountCode flag is enabled) Serverless: Injecting required Python packages to package... Type Error [ E R R_ I N V A L I D_ A R G_ T Y P E] ----- TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be one of type string, Buffer, or URL. Received type undefined at readFile (fs.js:296:3) at go$readFile (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\graceful-fs\graceful-fs.js:118:14) at Object.readFile (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\graceful-fs\graceful-fs.js:115:12) at Object.readFile (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\universalify\index.js:5:67) at injectRequirements (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\serverless-python-requirements\lib\inject.js:23:6) at ServerlessPythonRequirements.injectAllRequirements (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\serverless-python-requirements\lib\inject.js:120 :12) at ServerlessPythonRequirements.BbPromise.bind.then.then.then (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\serverless-python-requirements\index.js: 183:43) From previous event: at Object.after [as hook] (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\serverless-python-requirements\index.js:182:10) From previous event: at PluginManager.invoke (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\serverless\lib\classes\PluginManager.js:489:22) at PluginManager.spawn (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\serverless\lib\classes\PluginManager.js:509:17) at BbPromise.try (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\serverless\lib\plugins\deploy\deploy.js:122:50) From previous event: at Object.before:deploy:deploy [as hook] (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\serverless\lib\plugins\deploy\deploy.js:102:22) at BbPromise.reduce (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\serverless\lib\classes\PluginManager.js:489:55) From previous event: at PluginManager.invoke (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\serverless\lib\classes\PluginManager.js:489:22) at getHooks.reduce.then (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\serverless\lib\classes\PluginManager.js:524:24) From previous event: at PluginManager.run (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\serverless\lib\classes\PluginManager.js:524:8) at variables.populateService.then (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\serverless\lib\Serverless.js:131:33) From previous event: at Serverless.run (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\serverless\lib\Serverless.js:118:74) at serverless.init.then (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\serverless\bin\serverless.js:80:26) at runCallback (timers.js:705:18) at tryOnImmediate (timers.js:676:5) at processImmediate (timers.js:658:5) at process.topLevelDomainCallback (domain.js:126:23) From previous event: at Object. (G:\projects\parse\trash\examples\aws-python-line-echo-bot\node_modules\serverless\bin\serverless.js:80:4) at Module._compile (internal/modules/cjs/loader.js:778:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10) at Module.load (internal/modules/cjs/loader.js:653:32) at tryModuleLoad (internal/modules/cjs/loader.js:593:12) at Function.Module._load (internal/modules/cjs/loader.js:585:3) at Function.Module.runMain (internal/modules/cjs/loader.js:831:12) at startup (internal/bootstrap/node.js:283:19) at bootstrapNodeJSCore (internal/bootstrap/node.js:622:3) Get Support -------------------------------------------- Docs: docs.serverless.com Bugs: github.com/serverless/serverless/issues Issues: forum.serverless.com Your Environment Information --------------------------- Operating System: win32 Node Version: 10.16.3 Framework Version: 1.73.1 Plugin Version: 3.6.13 SDK Version: 2.3.1 Components Version: 2.31.0 ``` ┆Issue is synchronized with this [Jira Task](https://localstack.atlassian.net/browse/LOC-31) by [Unito](https://www.unito.io/learn-more)

[maranqz]

Problem with serverless-python-requirements, it's trying add requirements to unexist artifacte. For resolve it by hack need to use

pythonRequirements:
  layer: true
# or
pythonRequirements:
  zip: true

For resolving by serverless-localstack need to extend skipIfMountLambda for override private js function like that. https://github.com/UnitedIncome/serverless-python-requirements/blob/master/index.js#L166

[whummer]

Hi @maranqz , thanks for reporting. Good catch about the js function that would have to be overwritten. Do you think you could try and help create a pull request to add this missing functionality? That would be really appreciated! Thanks for your help.

[chris-erickson]

Any movement on this? It still doesn't seem possible to mount python code into Localstack? The above options yield layer issues for me, even doing all the serverless-python-requirements don't seem to help either:

    pythonRequirements:
        usePoetry: true
        dockerizePip: "non-linux"
        layer: true

    localstack:
        stages:
            - local
        host: http://localhost
        edgePort: 4566
        autostart: false
        lambda:
            mountCode: true
        docker:
            sudo: false
        debug: true

functions:
    hello:
        handler: app.lambda_handler
        ...
        layers:
            - Ref: PythonRequirementsLambdaLayer

Is it possible this is a limitation of the not-paid version of Localstack?

[srinivaskalyani]

Hi @whummer , gentle bump on this again. This problem still persists. I am still unable to mount the lambda code.

[whummer]

Hi @chris-erickson @srinivaskalyani, thanks for the update, and apologies for the delay. In addition to the serverless.yml, could you please share a copy of your startup configs (e.g., docker-compose.yml) with us, so we can fully reproduce this use case end-to-end? Thanks for your help!

[srinivaskalyani]

Hi @whummer , Thanks a lot for taking this up again and sorry for being a bother. I am using a local installation of Localstack, not the Docker container. I am sharing a copy of my serverless.yml. Please let me know if I must provide any further information.

service: thetenant-service
plugins:
  - serverless-python-requirements
  - serverless-pseudo-parameters
  - serverless-localstack

custom:
  pythonRequirements:
    dockerizePip: false
    layer: true
  localstack:
    stages:
      - local
    debug: true
    host: http://localhost
    edgePort: 4566
    region: us-east-1
    lambda:
      mountCode: true

package:
  exclude:
    - ./**
  include:
    - my_serverless_root/**

provider:  
  name: aws
  runstime: python3.7
  stage: ${opt:stage, 'default_stage'}
  tenant: thetenant
  region: us-east-1
  deploymentBucket: my-deployment-bucket
  role: arn:aws:iam::#{AWS::AccountId}:role/MyLambdaExecutionRole
  environment:
    STAGE: ${self:provider.stage}
    TENANT: ${self:provider.tenant}

functions:
  my_function:
    name: ${self:provider.tenant}-my-function
    handler: my_handler_dir.aws.my_handler_file.handler
    events:
      - http: POST /my_api/endpoint

[philihp]

Bump

[ptrhck]

Assuming that this is resolved and you can mount your code, I am still wondering if you will be able to run this code. Because, the idea of using serverless-python-requirements is to add some third-party Python libraries the code depends on. With mountCode: true only the source code will be mounted, packaging is skipped and, thus, no third-party libraries will be available at runtime of the lambda function.

Am I missing the part where third-party dependencies are considered when using local code mounting? This issue also points to that problem https://github.com/localstack/localstack/issues/6181

Can the packaging be forced, even with mounCode?

[steffyP]

Hi @ptrhck,

to best of my knowledge, the serverless-python-requirements is currently not considered, e.g. if you use the mountCode: true all dependencies should be in the directory that will be mounted.