search

rurri / serverless-resources-env

Serverlss framework plugin, which after a deploy, fetches cloudformation resource identifiers and sets them on AWS lambdas, and creates local ./.serverless-resources-env/.<region>_<stage>_<function-name> file
MIT License
20 stars 7 forks source link

Too many resources #1

Closed fersman closed 7 years ago

fersman commented 7 years ago

I have CloudFormation with 100 resources and I get this error when using serverless-resources-env plugin: "Request must be smaller than 5120 bytes for the UpdateFunctionConfiguration operation".

Written .env file has 6100 bytes. I think this is AWS limit.

rurri commented 7 years ago

Thanks for reporting this. Sounds like it might be a limit with the Update request and not the number of environment variables it can contain. If this is the case we can split up the update into multiple calls. Will investigate.

rurri commented 7 years ago

I am able to reproduce with the following serverless.yml

resources:
  Resources:
    testTopic1:
      Type: AWS::SNS::Topic
    testTopic2:
      Type: AWS::SNS::Topic
    testTopic3:
      Type: AWS::SNS::Topic
    testTopic4:
      Type: AWS::SNS::Topic
    testTopic5:
      Type: AWS::SNS::Topic
    testTopic6:
      Type: AWS::SNS::Topic
    testTopic7:
      Type: AWS::SNS::Topic
    testTopic8:
      Type: AWS::SNS::Topic
    testTopic9:
      Type: AWS::SNS::Topic
    testTopic10:
      Type: AWS::SNS::Topic
    testTopic11:
      Type: AWS::SNS::Topic
    testTopic12:
      Type: AWS::SNS::Topic
    testTopic13:
      Type: AWS::SNS::Topic
    testTopic14:
      Type: AWS::SNS::Topic
    testTopic15:
      Type: AWS::SNS::Topic
    testTopic16:
      Type: AWS::SNS::Topic
    testTopic17:
      Type: AWS::SNS::Topic
    testTopic18:
      Type: AWS::SNS::Topic
    testTopic19:
      Type: AWS::SNS::Topic
    testTopic20:
      Type: AWS::SNS::Topic
    testTopic21:
      Type: AWS::SNS::Topic
    testTopic22:
      Type: AWS::SNS::Topic
    testTopic23:
      Type: AWS::SNS::Topic
    testTopic24:
      Type: AWS::SNS::Topic
    testTopic25:
      Type: AWS::SNS::Topic
    testTopic26:
      Type: AWS::SNS::Topic
    testTopic27:
      Type: AWS::SNS::Topic
    testTopic28:
      Type: AWS::SNS::Topic
    testTopic29:
      Type: AWS::SNS::Topic
    testTopic30:
      Type: AWS::SNS::Topic
    testTopic31:
      Type: AWS::SNS::Topic
    testTopic32:
      Type: AWS::SNS::Topic
    testTopic33:
      Type: AWS::SNS::Topic
    testTopic34:
      Type: AWS::SNS::Topic
    testTopic35:
      Type: AWS::SNS::Topic
    testTopic36:
      Type: AWS::SNS::Topic
    testTopic37:
      Type: AWS::SNS::Topic
    testTopic38:
      Type: AWS::SNS::Topic
    testTopic39:
      Type: AWS::SNS::Topic
    testTopic40:
      Type: AWS::SNS::Topic
    testTopic41:
      Type: AWS::SNS::Topic
    testTopic42:
      Type: AWS::SNS::Topic
    testTopic43:
      Type: AWS::SNS::Topic
    testTopic44:
      Type: AWS::SNS::Topic
    testTopic45:
      Type: AWS::SNS::Topic
    testTopic46:
      Type: AWS::SNS::Topic
    testTopic47:
      Type: AWS::SNS::Topic
    testTopic48:
      Type: AWS::SNS::Topic
    testTopic49:
      Type: AWS::SNS::Topic
    testTopic50:
      Type: AWS::SNS::Topic
    testTopic51:
      Type: AWS::SNS::Topic
    testTopic52:
      Type: AWS::SNS::Topic
    testTopic53:
      Type: AWS::SNS::Topic
    testTopic54:
      Type: AWS::SNS::Topic
    testTopic55:
      Type: AWS::SNS::Topic
    testTopic56:
      Type: AWS::SNS::Topic
    testTopic57:
      Type: AWS::SNS::Topic
    testTopic58:
      Type: AWS::SNS::Topic
    testTopic59:
      Type: AWS::SNS::Topic
    testTopic60:
      Type: AWS::SNS::Topic
    testTopic61:
      Type: AWS::SNS::Topic
    testTopic62:
      Type: AWS::SNS::Topic
    testTopic63:
      Type: AWS::SNS::Topic
    testTopic64:
      Type: AWS::SNS::Topic
    testTopic65:
      Type: AWS::SNS::Topic
    testTopic66:
      Type: AWS::SNS::Topic
    testTopic67:
      Type: AWS::SNS::Topic
    testTopic68:
      Type: AWS::SNS::Topic
    testTopic69:
      Type: AWS::SNS::Topic
    testTopic70:
      Type: AWS::SNS::Topic
    testTopic71:
      Type: AWS::SNS::Topic
    testTopic72:
      Type: AWS::SNS::Topic
    testTopic73:
      Type: AWS::SNS::Topic
    testTopic74:
      Type: AWS::SNS::Topic
    testTopic75:
      Type: AWS::SNS::Topic
    testTopic76:
      Type: AWS::SNS::Topic
    testTopic77:
      Type: AWS::SNS::Topic
    testTopic78:
      Type: AWS::SNS::Topic
    testTopic79:
      Type: AWS::SNS::Topic
    testTopic80:
      Type: AWS::SNS::Topic
    testTopic81:
      Type: AWS::SNS::Topic
    testTopic82:
      Type: AWS::SNS::Topic
    testTopic83:
      Type: AWS::SNS::Topic
    testTopic84:
      Type: AWS::SNS::Topic
    testTopic85:
      Type: AWS::SNS::Topic
    testTopic86:
      Type: AWS::SNS::Topic
    testTopic87:
      Type: AWS::SNS::Topic
    testTopic88:
      Type: AWS::SNS::Topic
    testTopic89:
      Type: AWS::SNS::Topic
    testTopic90:
      Type: AWS::SNS::Topic
    testTopic91:
      Type: AWS::SNS::Topic
    testTopic92:
      Type: AWS::SNS::Topic
    testTopic93:
      Type: AWS::SNS::Topic
    testTopic94:
      Type: AWS::SNS::Topic
    testTopic95:
      Type: AWS::SNS::Topic
    testTopic96:
      Type: AWS::SNS::Topic
    testTopic97:
      Type: AWS::SNS::Topic
    testTopic98:
      Type: AWS::SNS::Topic
    testTopic99:
      Type: AWS::SNS::Topic
    testTopic100:
      Type: AWS::SNS::Topic
rurri commented 7 years ago

Turns out AWS replaces all environment variables on this request, so splitting these off into different requests does not fix the issue. It makes it so that it does not error, but then only some of the vars are set.

I am thinking for a case like this we will want to do something like allow a list of resources that we want exported as env variables to be defined on each function.

perhaps something like:

functions:
  hello:
    custom:
      env-resources:
        - testTopic1
        - testTopic3
        - testTopic10
        - testTopic11
rurri commented 7 years ago

New version now maps resources per function in the serverless.yml so that it doesn't run into that limit. (Well assuming you don't need all 100 resources in a single function.)

Currently in #master. Will be a part of 0.3.0 once released.

rurri commented 7 years ago

This is now on 0.3.0 and released to npm at that version.