Failure to work with the pseudo param plugin

[jack1902]

tl;dr pseudo param plugin and this plugin don't play nicely together :(

link to pseudo paramter plugin

So until recently, we didn't have 200 resources in a stack and all was sunshine and rainbows with the pseudo param plugin making horrific looking things like:

thing: !Join
  - ''
  - - "s3//"
    - !Ref Bucket
    - "/path/to/stuff" 

became

thing: "s3://#{Bucket}/path/to/stuff"

This was amazing until we added more resources and hit the 200 resource per stack limit.

We have a custom stacks-map.js with:

stacks-map.js

```js var root_stack = [ "ServerlessDeploymentBucket", "ServerlessDeploymentBucketPolicy", "RawGlueRole", "ProcessedGlueRole", "CommonClassifier", "RawBucket", "RawBucketPolicy", "ProcessedBucket", "RawGlueDB", "ProcessedGlueDB" ] module.exports = (resource, logicalId) => { if (root_stack.includes(logicalId)) return false ; if (logicalId.startsWith("ResourceGroupOne")) return { destination: 'ResourceGroupOne' }; if (logicalId.startsWith("Caller")) return { destination: 'Caller' }; console.log('Resource: ${logicalId} doesn\'t have a home') return true // Falls back to default }; ```

This stack map allows us to break things down into their relevant stacks, but the problem is that this plugin isn't converting the pseudo param plugin #{} methods with their correspondent !Ref and !GetAtt, we want to keep our serverless looking as clean as possible as number !Joins cause files to become 100s of lines in length in no time at all.

Would it be possible to get support for the pseudo param plugin ref/getatt style added to this plugin so that we can use both together without having to make a tonne of changes to all of our current infrastructure to use !Ref !GetAtt approach in order for this plugin to do the param approach?

[dougmoscrop]

What order are the plugins in? That could be a first thing to check. Otherwise yes I would absolutely take PRs to fix this.

[jack1902]

What order are the plugins in? That could be a first thing to check. Otherwise yes I would absolutely take PRs to fix this.

Thanks for the fast response!

So this is the plugins section in our serverless.yml

plugins:
  - serverless-python-requirements
  - serverless-iam-roles-per-function
  - serverless-s3-remover
  - serverless-pseudo-parameters
  - serverless-plugin-split-stacks

I am not sure if the order of the above has any impact as my understanding is that you can use the hooks to carry out certain actions within a plugin, so you can say before:deploy etc. Is this ordered? since i would essentially need the pseudo param outputs to be replaced. ie

thing: "s3://${Bucket}/path/to/stuff"

becomes (within a nested stack)

parameter:
   bucketname:
     description: "name of the bucket (being used a place holder)"

resources:
  resourceInNestedStack:
    thing: s3://${bucketname}/path/to/stuff 

Same applies to the Resource.Att approach with the sub. How would i check what order the plugins are being executed? I am hoping that the pseudo param plugin is first, then we just have to do some replacements for the nested stuff to work appropriately

Edit: i am just looking at https://serverless.com/framework/docs/providers/google/guide/plugins/ to better understand the ordering

[dougmoscrop]

you're right about the hooks, in this case both of our plugins are hooking on the same event (after merge resources stuff) -- so the order then becomes the order they're defined in serverless.yml AFAIK.

The way I usually debug this is just sprinkling console.logs around and using serverless package to see what's going on

[jack1902]

So if my assumptions are correct, by having the param plugin first, it sets up one stack with every resource in it replacing the #{sub_version_of_resource} and #{sub_version_of_get_att.att} with the appropriate !Sub ${resource} and !Sub ${resource.att}.

So as long as param plugin is first, then this plugin would just need to convert the !Sub ${resource} into !Sub ${Parameter_passed_in_from_parent/root} etc. Not sure how hard a time you had with the original !Ref and !GetAtt stuff but it seems like an additional method to replace the !Sub and just have the value passed in as a parameter from the parent/root using the same method as !Ref currently but with support for !Sub ?

Happy to assist with this if this is the case

[dougmoscrop]

I think you're on the right track. In fact it's probably just poor reference detection (the ! Shorthand?) - sorry I am not trying to be terse I'm just in the middle of moving houses and crap.

I would debug around https://github.com/dougmoscrop/serverless-plugin-split-stacks/blob/e43117a8145ee661b024348d6f52d2858a89a2d4/lib/utils.js#L201 and see if maybe the issue is we need to do like full string value parsing or regex to detect these?

[JBrashko]

Did some work on trying to get it working https://github.com/Meliairon/serverless-plugin-split-stacks

[simonobe]

is this planned to be fixed and merged (and released) any time soon? we want to use this plugin, but this problem is preventing us from using it.

[dougmoscrop]

Help is definitely needed. Maybe pull @Meliairon branch and go from there?

[simonobe]

Help is definitely needed. Maybe pull @Meliairon branch and go from there?

@Meliairon what is the current status of your fork? Did you manage to get it to work? What work is there left?

[JBrashko]

I have managed to get a hacky version working that removes the blocker for my team but it is a very limited solution. I am looking at making a version of it that solves the problem properly but that is considerably more involved. At the moment it will turn anything with a sub refrence to an attribute into a seperate parameter and I am looking at how I can either use the sub function within the stack or split it up into a join array without it becoming a huge mess. Also I can think of a dozen ways off the top of my head that the hacky solution will break and rather than fix them I want to do it properly.

[jack1902]

Thanks for the fix, can we get a version of this plugin released with this fix in please?

[dougmoscrop]

yes, sorry