Open solarmosaic-kflorence opened 3 years ago
Here is a more thorough example. This is the project structure (this project exists in our monorepo repository at location: ./libraries/terraform/aws/bucket-object
):
.
├── main.tf
├── modules
│ └── policies
│ ├── main.tf
│ ├── outputs.tf
│ └── variables.tf
├── outputs.tf
├── readme.md
├── variables.tf
└── versions.tf
Inside main.tf
there is a module reference:
module "session" {
source = "s3::https://s3.amazonaws.com/<REDACTED>/terraform/terraform-aws-session/terraform-aws-session_v0.2.2.zip"
}
Starting with no .terraform
cache, I run terraform init
in the root of the project...
➜ monorepo git:(feature/foo) cd libraries/terraform/aws/bucket-object
➜ bucket-object git:(feature/foo) terraform init
Initializing modules...
Downloading s3::https://s3.amazonaws.com/<REDACTED>/terraform/terraform-aws-session/terraform-aws-session_v0.2.2.zip for session...
- session in .terraform/modules/session
Initializing the backend...
Initializing provider plugins...
- Reusing previous version of hashicorp/aws from the dependency lock file
- Installing hashicorp/aws v3.61.0...
- Installed hashicorp/aws v3.61.0 (signed by HashiCorp)
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
The project .terraform
cache folder now has the following structure:
.
├── modules
│ ├── modules.json
│ └── session
│ ├── main.tf
│ ├── modules
│ │ ├── meta
│ │ │ └── outputs.tf
│ │ └── policies
│ │ ├── main.tf
│ │ └── outputs.tf
│ ├── outputs.tf
│ └── versions.tf
└── providers
└── registry.terraform.io
└── hashicorp
└── aws
└── 3.61.0
└── darwin_amd64
└── terraform-provider-aws_v3.61.0_x5
The modules/modules.json
file has the following content:
{"Modules":[{"Key":"","Source":"","Dir":"."},{"Key":"session","Source":"s3::https://s3.amazonaws.com/<REDACTED>/terraform/terraform-aws-session/terraform-aws-session_v0.2.2.zip","Dir":".terraform/modules/session"}]}
IntelliJ now correctly resolves this module from the cache, so the modules cache at the root of the project work fine. Now let's resolve the module references for the ./modules/policies
sub-module, which contains the following in it's main.tf
file:
module "session" {
source = "s3::https://s3.amazonaws.com/<REDACTED>/terraform/terraform-aws-session/terraform-aws-session_v0.2.2.zip//modules/policies"
}
Again starting from having no .terraform
cache, I navigate into the sub-module folder and run terraform init
:
➜ bucket-object git:(feature/foo) cd modules/policies
➜ policies git:(feature/foo) terraform init
Initializing modules...
Downloading s3::https://s3.amazonaws.com/<REDACTED>/terraform/terraform-aws-session/terraform-aws-session_v0.2.2.zip for session...
- session in .terraform/modules/session/modules/policies
Initializing the backend...
Initializing provider plugins...
Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.
If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary.
The ./modules/policies/.terraform
cache directory now has the following structure:
.
└── modules
├── modules.json
└── session
├── main.tf
├── modules
│ ├── meta
│ │ └── outputs.tf
│ └── policies
│ ├── main.tf
│ └── outputs.tf
├── outputs.tf
└── versions.tf
The modules/modules.json
file has the following content:
{"Modules":[{"Key":"","Source":"","Dir":"."},{"Key":"session","Source":"s3::https://s3.amazonaws.com/<REDACTED>/terraform/terraform-aws-session/terraform-aws-session_v0.2.2.zip//modules/policies","Dir":".terraform/modules/session/modules/policies"}]}
However, IntelliJ cannot resolve the module:
Exactly the same problem.
terraform plan
works A-OK but PHPStorm doesn't find the module files
EDIT: This got fixed by just restarting the IDE.
I believe it worked with multiple terraform modules in same project before and now it can't locate modules for unknown reason for all but one module.
Not sure if it's the same cause but had the same issue with Terraform 1.1.0. They changed how they store the Source
in .terraform/modules/modules.json
causing my issue. See https://github.com/hashicorp/terraform/pull/28854. Looks like the plugin reads that file.
Fixed it by making sure the Source
field in .terraform/modules/modules.json
matches exactly the source
in the TF file. If doesn't match, modify it for all the modules in that file and restart the IDE (or close and open the file).
Example for an official module: change registry.terraform.io/terraform-aws-modules/vpc/aws
to terraform-aws-modules/vpc/aws
I'm using PyCharm 2021.3 and version 0.7.14 of the plugin. I have also a monorepo with multiple Terraform projects!
Same issue. This seems to have started recently:
Webstorm: Build #WS-213.5744.224, built on November 27, 2021 Plugin 0.7.14 OS: Arch Linux Error Cannot locate module locally: Unknown reason
module "gce-container-c1" {
source = "terraform-google-modules/container-vm/google"
version = "~> 2.0"
container = {
image = "eu.gcr.io/XXXX"
}
restart_policy = "Always"
}
Any workarounds?
Not sure if it's the same cause but had the same issue with Terraform 1.1.0. They changed how they store the
Source
in.terraform/modules/modules.json
causing my issue. See hashicorp/terraform#28854. Looks like the plugin reads that file.
I can confirm, this solved the issue for me. Since the original reporter used 1.0.10, what we experienced here is a separate issue
Alternativlity, you can prepend the source URI in your main.tf
with registry.terraform.io/
. Not including it is just a short hand way of indicating that the module is hosted on the Terraform registry. By prepending the registry in your source reference you don't have to touch the files in the cache.
module "vpc" {
source = "registry.terraform.io/terraform-aws-modules/vpc/aws"
version = "3.11.0"
}
Thanks @danpoland , it's really helpful!
looks like plugin successfully can find a remote modules after terraform get with terraform version 1.0.11 and below, unfortunately can't find modules with latest terraform versions.
@shuliakovsky see my reponse above in case you missed it for a workaround: https://github.com/VladRassokhin/intellij-hcl/issues/365#issuecomment-996019841.
You will to need to update the .terraform/modules/modules.json
for the plugin to work properly when using Terraform >= 1.1.0
.
We can use sed
to quickly fix a repository with multiple workspaces:
sed -i --regexp-extended "s/registry.terraform.io\///g" **/.terraform/modules/modules.json && sed -i --regexp-extended "s/git::https:\/\/(.*)\.git/\1/" **/.terraform/modules/modules.json
I set an alias to this command so I can easily rerun it.
Please see https://github.com/VladRassokhin/intellij-hcl/issues/373 for issues related to the module paths introduced in Terraform 1.1.x
-- the issue in this ticket only relates to sub-modules.
Would there be a way to update the plugin to look for the new Source address alongside the exact match?
From glancing at hashicorp/terraform#28854 I see the following:
registry/module
--> registry.terraform.io/registry/module
github.com/some/module
--> git::https://github.com/some/module
git@github.com:some/module
--> git::ssh://git@github.com/some/module
^By no means is this an exhaustive list, I compiled it from looking at some of the updated unit tests.
The fixes mentioned above do work for me, but it is a lot more verbose than I'd like. If the plugin can search modules.json for both an exact match OR an expected transformed match, wouldn't it handle the cases for both Terraform 1.1.x and below that as well?
I've just recently had this issue, again despite checking that the values under source =
and values under the file modules.json
are the same
I'm not sure why this happened, but the above steps seem to solve it temporarily
Not sure if it's the same cause but had the same issue with Terraform 1.1.0. They changed how they store the
Source
in.terraform/modules/modules.json
causing my issue. See hashicorp/terraform#28854. Looks like the plugin reads that file.Fixed it by making sure the
Source
field in.terraform/modules/modules.json
matches exactly thesource
in the TF file. If doesn't match, modify it for all the modules in that file and restart the IDE (or close and open the file).Example for an official module: change
registry.terraform.io/terraform-aws-modules/vpc/aws
toterraform-aws-modules/vpc/aws
I'm using PyCharm 2021.3 and version 0.7.14 of the plugin. I have also a monorepo with multiple Terraform projects!
I have same issue using:
This answer still works, thanks!
In my case I had to switch from terraform-aws-modules/vpc/aws
to registry.terraform.io/terraform-aws-modules/vpc/aws
.
In my case I had to switch from
terraform-aws-modules/vpc/aws
toregistry.terraform.io/terraform-aws-modules/vpc/aws
.
- terraform 1.2.4
- Intellij 2022.1.2
That tip was life-saving for me, thanks @lozuwa
I can confirm the issue is still there and the solution mentioned above works.
https://plugins.jetbrains.com/plugin/7808-terraform-and-hcl v222.4345.14 PhpStorm 2022.2.3 Terraform v1.3.3 on linux_amd64
Prerequisites
Installation details
Help->About->Copy to Clipboard
)Settings->Plugins
) -0.7.14
terraform -v
) -v1.0.10
(darwin_amd64
)Terraform Configuration Files
And here is the project folder (including
.terraform
cache):Exception
N/A
Expected Behavior
The HCL plugin should be able to reference any module cached in the
./.terraform
in the current directory. We are using Terraform inside of a monorepo, so we have several directories scattered around with Terraform configuration. I have noticed in some project folders we do not have this issue and in others we do. It seems to happen most often within sub-modules (e.g../modules/bar
, relative to the project root), or when referencing sub-folders of a module, for example:Actual Behavior
The module exists, of course, and is downloaded and cached in the
.terraform
folder. Performing Terraform CLI commands in the directory will succeed. It's just a problem with the plugin cache.In the example we are downloading the module from S3, but it shouldn't matter what the source is. Notably, though, I haven't seen this happen with a local module reference (e.g.
source = "../../path/to/module"
).Steps to Reproduce
See the comment below https://github.com/VladRassokhin/intellij-hcl/issues/365#issuecomment-981972598