Fine-grained hot reloading configuration for functions and layers to replace global `mountCode` option

[joe4dev]

The current hot reloading configuration mountCode has many limitations (e.g., global config, no layer support). An improved configuration API and documentation could make hot reloading more powerful and easier to use with the Serverless Framework.

Related Issues/PRs

• https://github.com/localstack/serverless-localstack/pull/222
• https://github.com/localstack/serverless-localstack/issues/129
• https://github.com/localstack/serverless-localstack/issues/216

Challenges

• mountCode: true is a global option but currently does not support hot-reloading of layers (new feature in LocalStack 2.0).
• Hot reloading (currently) only works with at most one layer per function. Hence, if we intend to apply mountCode to layers as well, it fails if there is any function with more than one layer in the serverless.yml
• mountCode is bad terminology because the current implementation does not do any mounting anymore but rather copies ZIP files (https://docs.localstack.cloud/user-guide/tools/lambda-tools/hot-reloading/). hotReload would be a better name
• We need to consider the scope of hot reloading because not everyone wants to enable hot reloading globally. Selectively enabling hot reloading for single functions or layers under development would be helpful.
• mountCode: true uses the current directory (assumed to be project directory) as default hot reload path or accepts relative paths (e.g., ./functions). This might lead to many unnecessary and slow reloads.

Background

• Serverless documentation on how to provide custom plugin configuration options: https://www.serverless.com/framework/docs/guides/plugins/custom-configuration
• It is possible to add custom function-level properties using "Function properties via defineFunctionProperties" via defineFunctionProperties
  • There is no well-defined mechanism to add layer-level properties. It might be possible to do manually using addPropertiesToSchema to add something like defineLayerProperties?! Might need to relax provider schema by allowing additional properties like for functions.patternProperties here https://github.com/serverless/serverless/blob/ed15cb27aee68954c93d875da96274914943ad71/lib/classes/config-schema-handler/index.js#L335
  • There is some API to extend configuration: https://www.serverless.com/framework/docs/guides/plugins/extending-configuration
• Serverless v3 comes with breaking changes regarding plugin variable resolving (e.g., comes with stricter validation): https://www.serverless.com/framework/docs/guides/upgrading-v3#new-variable-resolver-engine

User Stories

Enable hot reloading for:

1. Single function code
2. Single function code + single layer
3. Single layer
4. Two functions sharing the same hot-reloaded layer

These cases are supported by LocalStack but not by the serverless-localstack plugin.

Possible API

service: myService

plugins:
  - serverless-localstack

custom:
  localstack:
# DEPRECATED global config
#    lambda:
#      mountCode: true
    stages:
      - local
    host: http://127.0.0.1

provider:
  name: aws
  region: us-east-1
  runtime: python3.9
  # OPTIONAL provider-scoped config here (would need to think about function vs. layers scope)

# https://www.serverless.com/framework/docs/providers/aws/guide/layers
layers:
  layerOne:
    path: layerOne
    hotReload: true

# https://www.serverless.com/framework/docs/providers/aws/guide/functions
functions:
  hello:
    handler: handler.handler
    hotReload: true
    layers:
      - !Ref LayerOneLambdaLayer

Considerations

1. Hot reloading for layers is configured per layer (rather than per layer usage) and applies to all functions using the layer
2. LocalStack (currently) support at most one layer in total when using hot reloading for a layer. This makes it difficult to apply hot reloading globally for layers.
3. Should we consider supporting ZIP files for hot reloading such that we can re-use build steps from tools such as Serverless, Gradle, etc rather than adding extra steps to unzip these archives (Java Gradle example).

Alternatives to create layers with Serverless

Based on https://www.tutorialspoint.com/serverless/serverless_layer_creation.htm

1. Creating python-requirements layer
2. Using an existing layer from another function in the same region
3. Non-requirements/ generic layers (example: https://www.serverless.com/blog/publish-aws-lambda-layers-serverless-framework)

[dominikschubert]

Another user story for this would be having both image-based and zip-based lambda functions and we don't want the image-based functions to have the S3Uri etc. fields