Variable Resolution in v4 Requires Multiple Passes Unlike v3

[trend-bill-battushig]

Issue description

Description

After upgrading from Serverless Framework v3 to v4, the resolveVariable function now requires multiple passes to fully resolve variable references, which differs from v3's behavior where variables were resolved in a single pass.

Repro Steps:

serverless.yml

org: <redacted>
# "service" is the name of this project. This will also be added to your AWS resource names.
service: test-variable-resolution

provider:
  name: aws
  runtime: python3.12
  stage: ${opt:stage, 'dev'} # Use stage given in --stage or default to dev
  region: ${opt:region}

custom:
  configFileNameStage:
    dev: config.dev.yaml
    default: config.default.yaml
  stageConfigFileName: ${self:custom.configFileNameStage.${self:provider.stage}, self:custom.configFileNameStage.dev}
  stageConfig: ${file(${self:custom.stageConfigFileName})}
  defaultConfig: ${file(config.default.yaml)}

  test1: ${self:custom.stageConfig.test, self:custom.defaultConfig.test}

functions:
  hello:
    handler: handler.hello
    name: ${file(test.js):testResolution}

config.dev.yaml

test: test-dev-${self:provider.region}

config.default.yaml

test: test-default-${self:provider.region}

test.js

exports.testResolution = async ({ resolveVariable }) => {
  let test1 = await resolveVariable("self:custom.test1");
  console.log(test1);
  return test1
};

Output

${self:custom.stageConfig.test, self:custom.defaultConfig.test}

As we can see, the call to resolveVariable("self:custom.test1") returns a string that still contains nested references and it seems to require multiple calls until finally reaching the true resolved variable.

Context

Context

We observe this issue specifically in Serverless v4, whereas the same multi-level reference in v3 either resolved fully or did not exhibit the lazy, multi-step behavior as prominently.

In our projects, we often piece together stage-specific config values using nested references. Because of this regression in how resolveVariable() works, we are using a 'not so clean' workaround like the following:

while (typeof test1 === "string" && test1.includes("${")) {
    test1 = await resolveVariable(test1);
}

Serverless Version: v4.4.17

Thank you for looking into this! If you need any more details or want me to test any potential fixes, please let me know.

[czubocha]

Hey @trend-bill-battushig, thanks for reporting. We're looking into it.

[sabayneh1]

@czubocha I’m also experiencing this issue, and it’s causing similar challenges in my setup. I see you’re looking into it—thank you for addressing this! Looking forward to a resolution.

[guptavasu1213]

I'm experiencing this issue as well. Please let us know how we can fix it.

[czubocha]

Hey @trend-bill-battushig @sabayneh1 @guptavasu1213, we've released a fix for this issue in v4.5.1. Please give it a try and let us know if you still encounter any problems.

[trend-bill-battushig]

Hello @czubocha , I will try it out and let you know. Thank you very much for your time and help!