28mm / blast-radius

Interactive visualizations of Terraform dependency graphs using d3.js
https://28mm.github.io/blast-radius-docs/
MIT License
2.04k stars 257 forks source link

Getting 500 error for GET /graph.json #43

Open kc1116 opened 5 years ago

kc1116 commented 5 years ago

Hello, my infra graph is displaying properly with the CSV but I get this error for the graph.json endpoint. Here is the command I am running.

blast-radius --port 3000 --serve --module-depth 3

127.0.0.1 - - [26/Sep/2018 02:59:50] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [26/Sep/2018 02:59:54] "GET /graph.svg HTTP/1.1" 200 -
[2018-09-26 02:59:57,836] ERROR in app: Exception on /graph.json [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.6/site-packages/blastradius/server/server.py", line 57, in graph_json
    tf = Terraform(os.getcwd())
  File "/usr/local/lib/python3.6/site-packages/blastradius/handlers/terraform.py", line 67, in __init__
    self.modules[name] = Terraform(directory=self.directory+'/'+source, settings=mod)
  File "/usr/local/lib/python3.6/site-packages/blastradius/handlers/terraform.py", line 26, in __init__
    self.config = hcl.load(config_io)
  File "/usr/local/lib/python3.6/site-packages/hcl/api.py", line 51, in load
    return loads(fp.read())
  File "/usr/local/lib/python3.6/site-packages/hcl/api.py", line 62, in loads
    return HclParser().parse(s)
  File "/usr/local/lib/python3.6/site-packages/hcl/parser.py", line 307, in parse
    return self.yacc.parse(s, lexer=Lexer())
  File "/usr/local/lib/python3.6/site-packages/ply/yacc.py", line 331, in parse
    return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)
  File "/usr/local/lib/python3.6/site-packages/ply/yacc.py", line 1199, in parseopt_notrack
    tok = call_errorfunc(self.errorfunc, errtoken, self)
  File "/usr/local/lib/python3.6/site-packages/ply/yacc.py", line 193, in call_errorfunc
    r = errorfunc(token)
  File "/usr/local/lib/python3.6/site-packages/hcl/parser.py", line 300, in p_error
    raise ValueError(msg)
ValueError: Line 131, column 4912: unexpected RIGHTBRACE
127.0.0.1 - - [26/Sep/2018 02:59:57] "GET /graph.json HTTP/1.1" 500 -
filiy commented 5 years ago

I'm getting the same error, any update on this?

127.0.0.1 - - [13/Nov/2018 08:48:44] "GET /favicon.ico HTTP/1.1" 404 -
127.0.0.1 - - [13/Nov/2018 08:48:46] "GET /graph.svg HTTP/1.1" 200 -
[2018-11-13 08:48:47,784] ERROR in app: Exception on /graph.json [GET]
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1718, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
    raise value
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/usr/local/lib/python3.6/site-packages/blastradius/server/server.py", line 57, in graph_json
    tf = Terraform(os.getcwd())
  File "/usr/local/lib/python3.6/site-packages/blastradius/handlers/terraform.py", line 67, in __init__
    self.modules[name] = Terraform(directory=self.directory+'/'+source, settings=mod)
  File "/usr/local/lib/python3.6/site-packages/blastradius/handlers/terraform.py", line 26, in __init__
    self.config = hcl.load(config_io)
  File "/usr/local/lib/python3.6/site-packages/hcl/api.py", line 51, in load
    return loads(fp.read())
  File "/usr/local/lib/python3.6/site-packages/hcl/api.py", line 62, in loads
    return HclParser().parse(s)
  File "/usr/local/lib/python3.6/site-packages/hcl/parser.py", line 307, in parse
    return self.yacc.parse(s, lexer=Lexer())
  File "/usr/local/lib/python3.6/site-packages/ply/yacc.py", line 331, in parse
    return self.parseopt_notrack(input, lexer, debug, tracking, tokenfunc)
  File "/usr/local/lib/python3.6/site-packages/ply/yacc.py", line 1199, in parseopt_notrack
    tok = call_errorfunc(self.errorfunc, errtoken, self)
  File "/usr/local/lib/python3.6/site-packages/ply/yacc.py", line 193, in call_errorfunc
    r = errorfunc(token)
  File "/usr/local/lib/python3.6/site-packages/hcl/parser.py", line 300, in p_error
    raise ValueError(msg)
ValueError: Line 564, column 16530: unexpected RIGHTBRACE
28mm commented 5 years ago

@filiy

Maybe we can narrow down the line # where the error is encountered (the line # given above isn't accurate). Several reports of this recently, so I may try to improve the error output.

Presumes pyhcl is installed:

pip3 install pyhcl

Then from a script or python shell:

from glob import iglob
import hcl

for filename in iglob('**.tf'):
    with open(filename, 'r') as f:
        print(filename)
        hcl.loads(f.read())
filiy commented 5 years ago

@28mm Excuse my python noobness i'm getting ImportError: No module named hcl not sure if that's me doing something wrong, or just something is broken.

pip3 install pyhcl
Requirement already satisfied: pyhcl in /usr/local/lib/python3.6/site-packages
Requirement already satisfied: ply==3.10 in /usr/local/lib/python3.6/site-packages (from pyhcl)

cat script.py

#!/usr/bin/env python
from glob import iglob
import hcl
for filename in iglob('**.tf'):
    with open(filename, 'r') as f:
        print(filename)
        hcl.loads(f.read())

./script.py

Traceback (most recent call last):
  File "./script.py", line 3, in <module>
    import hcl
ImportError: No module named hcl

System details

python -V
Python 2.7.15

pip -V
pip 9.0.3 from /usr/lib/python2.7/site-packages (python 2.7)

uname -r
4.18.16-100.fc27.x86_64

cat fedora-release 
Fedora release 27 (Twenty Seven)
28mm commented 5 years ago

@filiy

python 2.7 doesn't have access to the hcl module since it was installed with pip3. Easy to remedy, though: replace the shebang line in your script.py with:

#!/usr/bin/env python3
filiy commented 5 years ago

@28mm Cool. that worked.

$ ./script.py 
main.tf
output.tf
vars.tf
provider.tf
backend.tf
28mm commented 5 years ago

@filiy

I was expecting the parser to raise an exception while parsing one of those files. That plainly didn't happen. Does this code reference any external modules?

Also, if this configuration isn't sensitive, I'd be happy to troubleshoot on my system. My email is patrick.mcmurchie@gmail.com, if you are able to make a copy available to me.

filiy commented 5 years ago

@28mm Ahh, this does seem related to a certain tf config. I tested blast-radius with a simple tf config and it's working fine. Unfortunately I can't share the config. Here is a provider output and attached a copy from terraform graph which may help. It's a bit of complex config.

[phil@c3po terraform-origin-3.11]$ terraform providers
.
├── provider.azurerm
└── module.azure-origin-311
    ├── provider.azurerm (inherited)
    ├── provider.local
    ├── provider.null
    ├── provider.random
    └── provider.template

graph

filiy commented 5 years ago

@28mm Blast Radius getting some air time! Nice! https://youtu.be/-gKTeT3BgHE?t=383

tim-steinkuhler commented 4 years ago

Hi! I ran into the same problem just now, eventually I found that changing "${var.aws_access_key}" to var.aws_access_key caused the error somehow.

scarolan commented 4 years ago

I've noticed this happening on Terraform 0.12 code, which has unquoted variables as @tim-steinkuhler mentioned. It renders the graph but there are no colors and you can't drag it around.

ThiamSoon commented 4 years ago

I'm getting this issue as well. I read through the comments, but still could not get it to work. @filiy what did you change in your config for it to work?

ThiamSoon commented 4 years ago

my errors,

[2020-05-13 16:08:58,561] ERROR in app: Exception on /graph.json [GET] Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2447, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1952, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1821, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise raise value File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1950, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1936, in dispatch_request return self.view_functionsrule.endpoint File "/usr/local/lib/python3.7/site-packages/blastradius/server/server.py", line 61, in graph_json tf = Terraform(os.getcwd()) File "/usr/local/lib/python3.7/site-packages/blastradius/handlers/terraform.py", line 67, in init self.modules[name] = Terraform(directory=self.directory+'/'+source, settings=mod) File "/usr/local/lib/python3.7/site-packages/blastradius/handlers/terraform.py", line 26, in init self.config = hcl.load(config_io) File "/usr/local/lib/python3.7/site-packages/hcl/api.py", line 62, in load return loads(fp.read(), export_comments=export_comments) File "/usr/local/lib/python3.7/site-packages/hcl/api.py", line 81, in loads return HclParser().parse(s, export_comments=export_comments) File "/usr/local/lib/python3.7/site-packages/hcl/parser.py", line 643, in parse s, lexer=Lexer(export_comments=export_comments), debug=DEBUG File "/usr/local/lib/python3.7/site-packages/hcl/ply/yacc.py", line 503, in parse tok = self.errorfunc(errtoken) File "/usr/local/lib/python3.7/site-packages/hcl/parser.py", line 634, in p_error raise ValueError(msg) ValueError: Line 2, column 62: unexpected ASTERISK_PERIOD; expected COMMA, IDENTIFIER, STRING, COMMENT, MULTICOMMENT, MINUS, NUMBER, FLOAT, $end, RIGHTBRACE, QMARK, EQ, NE, LT, GT, LE, GE, ADD, MULTIPLY, DIVIDE, LEFTBRACKET, LEFTPAREN 127.0.0.1 - - [13/May/2020 16:08:58] "GET /graph.json HTTP/1.1" 500 -