When we try to destroy a stack the FindWhereWorkingDirIsIncluded() method is called which works its way from deepest to shallowest path searching recursively for all modules. This in turn checks the same modules multiple times. When there are lots of modules the repetitive loading alone (with no dependencies) takes almost an hour.
To Reproduce
Create hierarchy of terraform configs (see folder structure above) with >3K stacks in the project/pipeline layer.
Run terragrunt destroy from one of the project configs
Expected behavior
When running versions older that 0.51.9 the job finishes in seconds rather than any newer build it takes almost an hour.
Versions
Terragrunt version: >= 0.51.9
Terraform version: *
Environment details: Found on MacOS 14.3.1
Underlying Problem
After reviewing the code the issue is related to how the directories are scanned recursively repeatedly from lowest to highest checking for full dependencies of each module. This creates lots of redundancy, especially in our case, due to the fact that if there are no dependencies there is no storage of already checked modules to short circuit on.
Suggested Solution
My opinion is the least intrusive option, but also likely a bit of a cheap hack, is to add a flag that allows skipping the check on destroy as a way of saying I'm good with the destroy so don't bother recursively checking for dependencies.
Describe the bug We have a deeply nested hierarchy of terragrunt config modules.
When we try to destroy a stack the
FindWhereWorkingDirIsIncluded()
method is called which works its way from deepest to shallowest path searching recursively for all modules. This in turn checks the same modules multiple times. When there are lots of modules the repetitive loading alone (with no dependencies) takes almost an hour.To Reproduce
terragrunt destroy
from one of the project configsExpected behavior When running versions older that 0.51.9 the job finishes in seconds rather than any newer build it takes almost an hour.
Versions
Underlying Problem After reviewing the code the issue is related to how the directories are scanned recursively repeatedly from lowest to highest checking for full dependencies of each module. This creates lots of redundancy, especially in our case, due to the fact that if there are no dependencies there is no storage of already checked modules to short circuit on.
Suggested Solution My opinion is the least intrusive option, but also likely a bit of a cheap hack, is to add a flag that allows skipping the check on destroy as a way of saying I'm good with the destroy so don't bother recursively checking for dependencies.