Handle Remote Modules and Fix HCL Parsing

[Jrc356]

This does a variety of things that hopefully help out.

Change from using pyhcl to using python-hcl2

From my understanding terraform, by default, uses hcl2 now. The readme of pyhcl mentions that it does not support hcl2 (and therefore modern terraform). Changing to hcl2 seemed to clear up a lot of the unexpected X errors as well. However, there's still a goofy one when using : in place of = such as this:

app_settings = {
    "SOME_VAR": "SOME VALUE"
}

but this is an easy fix to just change the colons to equals

Improve error handling when parsing hcl

I changed up the way the tf files are loaded so that each file is loaded individually rather than creating a big combined one and then saving that to the Terraform.config. This way, we can throw an exception that's a little more indicative of where the problem is.

For example, if my file for the issue with colons described above is called myfunc.tf then when that is loaded, the exception thrown will read:

Traceback (most recent call last):
...
RuntimeError: Exception occurred while parsing /path/to/myfunc.tf

Use os.path.join

Noticed a comment in terraform.py about using path.join instead of the concat by addition so I changed that up

Handle remote module

Probably the pièce de résistance of this PR. Added the ability to handle remote modules (at least I think this should handle most cases). Since we require the terraform config to be init'd, we can assume all remote modules are found under .terrform of the topmost dir of the config. From here it's just about getting the right path out of the source description which a simple enough regex does the trick to split things. I tried this on a handful of sources including:

• local modules (just to make sure nothing broke)
• terraform registry
• git using ssh
• github.com

Make viewer usable on parsing failure

Even if the server fails, I personally want to be able to move around the graph it creates even if I don't get the full functionality, so I did just that. When the server fails to parse, the chart is still semi usable and an alert is added to the navbar to indicate that a server exception occurred

[nicholasmhughes]

I can confirm that this fixes my immediate "broken-ness" using blastradius with TF 0.12.26

[Jrc356]

I can confirm that this fixes my immediate "broken-ness" using blastradius with TF 0.12.26

Awesome to hear! Did you have an open issue for what was broken? If so, can you link it here please? If not, could you elaborate on the "broken-ness" a little please? This way we can gather a little on what this fixes since I included quite a bit into this PR

[nicholasmhughes]

Same as some other issues floating around... UNEXPECTED <INSERT THING HERE> in parsing TF files.

[2020-09-18 15:12:51,269] ERROR in app: Exception on /graph.json [GET]
Traceback (most recent call last):
  File "/home/nmhugh2/dev/tf-rks/envs/prod/rancher/env/lib64/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/nmhugh2/dev/tf-rks/envs/prod/rancher/env/lib64/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/nmhugh2/dev/tf-rks/envs/prod/rancher/env/lib64/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/nmhugh2/dev/tf-rks/envs/prod/rancher/env/lib64/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/nmhugh2/dev/tf-rks/envs/prod/rancher/env/lib64/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/nmhugh2/dev/tf-rks/envs/prod/rancher/env/lib64/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/nmhugh2/dev/tf-rks/envs/prod/rancher/env/lib64/python3.7/site-packages/blastradius/server/server.py", line 61, in graph_json
    tf = Terraform(os.getcwd())
  File "/home/nmhugh2/dev/tf-rks/envs/prod/rancher/env/lib64/python3.7/site-packages/blastradius/handlers/terraform.py", line 26, in __init__
    self.config = hcl.load(config_io)
  File "/home/nmhugh2/dev/tf-rks/envs/prod/rancher/env/lib64/python3.7/site-packages/hcl/api.py", line 62, in load
    return loads(fp.read(), export_comments=export_comments)
  File "/home/nmhugh2/dev/tf-rks/envs/prod/rancher/env/lib64/python3.7/site-packages/hcl/api.py", line 81, in loads
    return HclParser().parse(s, export_comments=export_comments)
  File "/home/nmhugh2/dev/tf-rks/envs/prod/rancher/env/lib64/python3.7/site-packages/hcl/parser.py", line 643, in parse
    s, lexer=Lexer(export_comments=export_comments), debug=DEBUG
  File "/home/nmhugh2/dev/tf-rks/envs/prod/rancher/env/lib64/python3.7/site-packages/hcl/ply/yacc.py", line 503, in parse
    tok = self.errorfunc(errtoken)
  File "/home/nmhugh2/dev/tf-rks/envs/prod/rancher/env/lib64/python3.7/site-packages/hcl/parser.py", line 634, in p_error
    raise ValueError(msg)
ValueError: Line 9, column 245: unexpected QMARK; expected COMMA, IDENTIFIER, STRING, COMMENT, MULTICOMMENT, MINUS, NUMBER, FLOAT, $end, RIGHTBRACE, RIGHTBRACKET, COLON, RIGHTPAREN

[Jrc356]

This is now 2 years stale, closing this