search

dependabot / dependabot-core

🤖 Dependabot's core logic for creating update PRs.
https://docs.github.com/en/code-security/dependabot
MIT License
4.7k stars 1.02k forks source link

Add support for Terraform 0.12 / HCL2 #2110

Closed marocchino closed 4 years ago

marocchino commented 5 years ago

Add support for Terraform 0.12 and HCL2 syntax. Dependabot currently fails with a cryptic error message when HCL2 syntax is parsed: 

Dependabot couldn't parse the main.tf found at `/terraform/manifests/xxx/main.tf`.

You can mention @dependabot in the comments below to contact the Dependabot team.
feelepxyz commented 5 years ago

@marocchino are you by any chance using Terraform 0.12? We currently don't support it and don't have a good way of detecting the version being used 😢

marocchino commented 5 years ago

@feelepxyz hmm. Actually, our project is using Terraform 0.12. and, Our main.tf contains below setting:

terraform {
  required_version = ">= 0.12.0"
  backend "local" {}
}

There seems to be no way to see the log, I'll take the core and run it locally and tell the result.

marocchino commented 5 years ago

Sorry for the newbie question. But, How can I point a private repo? Solved it.

marocchino commented 5 years ago

It looks like we need to send additional parameters for terraform.

1 

$ bin/dry-run.rb terraform vega-corporation/solomon-gamigin --dir=terraform/manifests/vpc
warning: parser/current is loading parser/ruby26, which recognizes
warning: 2.6.4-compliant syntax, but you are running 2.6.2.
warning: please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
=> fetching dependency files
=> dumping fetched dependency files: ./dry-run/vega-corporation/solomon-gamigin/terraform/manifests/vpc
Initialized empty Git repository in /home/dependabot/dependabot-core/dry-run/vega-corporation/solomon-gamigin/terraform/manifests/vpc/.git/

*** Please tell me who you are.

Run

  git config --global user.email "you@example.com"
  git config --global user.name "Your Name"

to set your account's default identity.
Omit --global to set the identity only in this repository.

fatal: unable to auto-detect email address (got 'dependabot@21be5d37e14a.(none)')
=> parsing dependency files
Traceback (most recent call last):
    12: from bin/dry-run.rb:283:in `
'
    11: from bin/dry-run.rb:185:in `cached_read'
    10: from bin/dry-run.rb:283:in `block in 
'
     9: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:24:in `parse'
     8: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:24:in `each'
     7: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:25:in `block in parse'
     6: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:222:in `parsed_file'
     5: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:34:in `in_a_temporary_directory'
     4: from /usr/lib/ruby/2.6.0/tmpdir.rb:93:in `mktmpdir'
     3: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:37:in `block in in_a_temporary_directory'
     2: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:37:in `chdir'
     1: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:37:in `block (2 levels) in in_a_temporary_directory'
/home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:231:in `block in parsed_file': unable to parse HCL: At 35:26: Unknown token: 35:26 IDENT var.vpc_name (Dependabot::SharedHelpers::HelperSubprocessFailed)
    7: from bin/dry-run.rb:283:in `
'
    6: from bin/dry-run.rb:185:in `cached_read'
    5: from bin/dry-run.rb:283:in `block in 
'
    4: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:24:in `parse'
    3: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:24:in `each'
    2: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:25:in `block in parse'
    1: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:219:in `parsed_file'
/home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:245:in `rescue in parsed_file': unable to parse HCL: At 35:26: Unknown token: 35:26 IDENT var.vpc_name (Dependabot::DependencyFileNotParseable)
2 

$ bin/dry-run.rb terraform vega-corporation/solomon-gamigin --dir=terraform/manifests/eks
warning: parser/current is loading parser/ruby26, which recognizes
warning: 2.6.4-compliant syntax, but you are running 2.6.2.
warning: please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.
=> fetching dependency files
=> dumping fetched dependency files: ./dry-run/vega-corporation/solomon-gamigin/terraform/manifests/eks
Reinitialized existing Git repository in /home/dependabot/dependabot-core/dry-run/vega-corporation/solomon-gamigin/terraform/manifests/eks/.git/

*** Please tell me who you are.

Run

  git config --global user.email "you@example.com"
  git config --global user.name "Your Name"

to set your account's default identity.
Omit --global to set the identity only in this repository.

fatal: unable to auto-detect email address (got 'dependabot@21be5d37e14a.(none)')
=> parsing dependency files
Traceback (most recent call last):
    12: from bin/dry-run.rb:283:in `
'
    11: from bin/dry-run.rb:185:in `cached_read'
    10: from bin/dry-run.rb:283:in `block in 
'
     9: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:24:in `parse'
     8: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:24:in `each'
     7: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:25:in `block in parse'
     6: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:222:in `parsed_file'
     5: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:34:in `in_a_temporary_directory'
     4: from /usr/lib/ruby/2.6.0/tmpdir.rb:93:in `mktmpdir'
     3: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:37:in `block in in_a_temporary_directory'
     2: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:37:in `chdir'
     1: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:37:in `block (2 levels) in in_a_temporary_directory'
/home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:231:in `block in parsed_file': unable to parse HCL: At 36:31: Unknown token: 36:31 IDENT data.terraform_remote_state.vpc.outputs.cluster_name (Dependabot::SharedHelpers::HelperSubprocessFailed)
    7: from bin/dry-run.rb:283:in `
'
    6: from bin/dry-run.rb:185:in `cached_read'
    5: from bin/dry-run.rb:283:in `block in 
'
    4: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:24:in `parse'
    3: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:24:in `each'
    2: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:25:in `block in parse'
    1: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:219:in `parsed_file'
/home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:245:in `rescue in parsed_file': unable to parse HCL: At 36:31: Unknown token: 36:31 IDENT data.terraform_remote_state.vpc.outputs.cluster_name (Dependabot::DependencyFileNotParseable)

We keep the tfvar and tfstate file in a separate directory and use environment variables to determine which environment values ​​to read.

$ terraform plan -state "../../data/<module_name>/<env_name>.tfstate" \
  -var-file="../../data/<env_name>.tfvars" \
  -var-file="../../data/<module_name>/common.tfvars" \
  -var-file="../../data/<module_name>/<env_name>.tfvars"
feelepxyz commented 5 years ago 
terraform {
  required_version = ">= 0.12.0"
  backend "local" {}
}

Ah nice one, we should be able to use this to return a better error. We put supporting terraform 0.12.0 on pause as we couldn't find a good way to parse the new HCL syntax. Looks like this is still experimental https://github.com/hashicorp/hcl2

rebelagentm commented 5 years ago

@feelepxyz I saw this PR (https://github.com/dependabot/dependabot-core/pull/1388) came in earlier today. Will this address the issue here?

feelepxyz commented 5 years ago

@rebelagentm yes possibly! @hmarr would love your 👀 on the above PR when you get a mo.

edahlseng commented 5 years ago

HCL 2 is no longer experimental! https://github.com/hashicorp/hcl/wiki/Version-Selection

hazcod commented 4 years ago

Copying from other issue:

Issue: https://github.com/ironPeakServices/infrastructure/issues/17 File in question: https://github.com/ironPeakServices/infrastructure/blob/master/main.tf

Terraform version:

terraform {
    required_version = ">= v0.12.19"
}
infin8x commented 4 years ago

Duplicate of #1176