search

amplify-education / serverless-vpc-discovery

Serverless plugin for discovering VPC / Subnet / Security Group configuration by name.
MIT License
38 stars 15 forks source link

VPC lookup happens for every lambda in serial and takes a long time for large projects #76

Closed tylerwalts closed 11 months ago

tylerwalts commented 1 year ago

On an existing large (~75 endpoints) serverless framework project we moved it to a VPC which is managed outside of serverless.

This plugin works to get the job done, but it is much slower than before since the lookup now seems to happen in series for every function in the project.

Is there a way to get it to only lookup the VPC once, and then re-use the value for all the rest?

jacobmaurer commented 1 year ago

FWIW - we're seeing the same bottleneck with our project with a good number of endpoints - it takes quite a while to discover each one sequentially. Perhaps it could be updated with a configurable option to discover them in parallel instead?

jacobmaurer commented 1 year ago

After poking around a bit, I think all it would take is a Promise.all() wrapper around https://github.com/amplify-education/serverless-vpc-discovery/blob/main/src/index.ts#L116, but for more controlled and configured concurrency, obviously a bit more involved (could use bluebird map with concurrency or equivalent)