serverless-plugin-split-stacks wont execute

mrossetto commented 6 months ago

Issue:

We're implementing the "serverless-plugin-split-stacks" to address the 500 resource limit by splitting our stack based on resource type. However, despite integrating it into our serverless.yml file, the plugin seems to be inactive, behaving as if it were not installed. We've meticulously checked our configurations, even replicating working setups from other stacks, yet the plugin remains overlooked.

Despite utilizing the --verbose output option, the logs show nothing unusual. It appears as though the plugin is absent, even though it is listed in the package registry, the plugin list, and configured correctly based off of the documentation.

Is there any additional insight or troubleshooting we can do to get this functioning or find the root cause to what is going with this?

Some of our troubleshooting:

Introducing deploymentMethod: direct had no effect.
Adjusting npm package versions didn't resolve the issue.
Reordering plugins in serverless.yml and exclusively only using serverless-plugin-split-stacks proved unsuccessful.
- Various configurations within the splitStacks section were explored without success. We tried using the stacks-map.js as well with no success.
- Replicating the exact setup from a functioning stack we have serverless-plugin-split-stacks in, including identical package versions and Serverless configurations, did not yield success in resolving the issue.
- Adding force: true did not yield any changes.
- We reached out to AWS; however, they don't have any more visibility on it than we do. They also mentioned a lot of our troubleshooting is what they would've advised.

Serverless.yml snippet:

frameworkVersion: "3"

custom:
  splitStacks:
    perFunction: false
    perType: true
    perGroupFunction: false

provider:
  name: aws
  deploymentMethod: direct
  runtime: nodejs16.x

plugins:
  - serverless-webpack
  - serverless-plugin-split-stacks
  - serverless-plugin-resource-tagging

package:
  individually: true

Packages:

"devDependencies": {
    "@babel/core": "^7.19.0",
    "@babel/eslint-parser": "^7.18.9",
    "@babel/preset-env": "^7.19.0",
    "babel-loader": "^8.2.5",
    "eslint": "^8.23.1",
    "jest": "^29.0.3",
    "nodemon": "^2.0.19",
    "npx": "^10.2.2",
    "serverless": "^3.38.0",
    "serverless-plugin-resource-tagging": "^1.2.0",
    "serverless-plugin-split-stacks": "^1.13.0",
    "serverless-s3-deploy": "^0.10.1",
    "serverless-webpack": "^5.9.1",
    "thread-loader": "^4.0.2",
    "webpack": "^5.74.0",
    "webpack-node-externals": "^3.0.0"
  },
  "dependencies": {
    "@aws-sdk/client-dynamodb": "^3.363.0",
    "@aws-sdk/client-eventbridge": "^3.347.0",
    "@aws-sdk/client-firehose": "^3.523.0",
    "@aws-sdk/client-lambda": "^3.398.0",
    "@aws-sdk/client-s3": "^3.347.0",
    "@aws-sdk/client-secrets-manager": "^3.347.0",
    "@aws-sdk/client-sqs": "^3.398.0",
    "@aws-sdk/client-ssm": "^3.347.0",
    "@aws-sdk/lib-dynamodb": "^3.365.0",
    "@aws-sdk/util-dynamodb": "^3.451.0",
    "@okta/jwt-verifier": "^3.0.1",
    "aws-sdk": "^2.1355.0",
    "axios": "^1.4.0",
    "debug": "^4.3.4",
    "jsonwebtoken": "^8.5.1",
    "jwt-decode": "^3.1.2",
    "lambda-local": "^2.1.2",
    "luxon": "^3.4.3",
    "mongodb": "^5.1.0",
    "stream-chain": "^2.2.5",
    "stream-json": "^1.8.0",
    "xml2js": "^0.6.0",
    "xmldom": "^0.6.0"
  }

dougmoscrop commented 6 months ago

Is this an existing deployment or a new stage?

mrossetto commented 6 months ago

Is this an existing deployment or a new stage?

This is occurring in an existing development stack.

dougmoscrop commented 6 months ago

So you'd need to be using a custom stacks map and force: true - can you try that again and is your stacks-map being called? At this point all you can really do is just add some console.log there and in this plugin in node_modules to debug it; I really can't stress how doing this to an existing stack is like surgery and very risky. 500 resources is huge and if at all possible, its' better to just start splitting your app up, even if it's just along stateful/stateless resource types.

mrossetto commented 6 months ago

So you'd need to be using a custom stacks map and force: true - can you try that again and is your stacks-map being called?

Thanks for the insight on all of that, and we were able to get that working. We ended up spinning up fresh stage with the same code and configuration, and it worked perfectly fine. Now we are testing different stacks-map.js configurations, but running into some circular dependencies between resources with the stacks. Hopefully we can get those circular dependencies ironed out.

dougmoscrop / serverless-plugin-split-stacks

serverless-plugin-split-stacks wont execute #214