Handling (removed) modules

[AnthonyWC]

I ran into this issue which I think has to do with the specify svg graph as I wasn't able to open the svg with certain program like GIMP (but can open with other program and terraform graph itself doesn't show error). I don't know much about svg to know why it'd cause such error.

terraform graph | blast-radius --svg > graph.svg                          
Traceback (most recent call last):
  File "/Users/antho/Library/Python/3.6/bin//blast-radius", line 95, in <module>
    main()
  File "/Users/antho/Library/Python/3.6/bin//blast-radius", line 53, in main
    dot = DotGraph('', file_contents=sys.stdin.read())
  File "/Users/antho/Library/Python/3.6/lib/python/site-packages/blastradius/handlers/dot.py", line 56, in __init__
    self.nodes.append(DotNode(e.target))
  File "/Users/antho/Library/Python/3.6/lib/python/site-packages/blastradius/handlers/dot.py", line 414, in __init__
    self.module         = DotNode._module(label) # for module groupings. 'root' or 'module.foo.module.bar'
  File "/Users/antho/Library/Python/3.6/lib/python/site-packages/blastradius/handlers/dot.py", line 448, in _module
    return m.groupdict()['module']
AttributeError: 'NoneType' object has no attribute 'groupdict'```

[28mm]

Hi @AnthonyWC

Okay -- I think I have a better sense of what is going on. I've made a mistaken assumption in the regex responsible for identifying module names. This is the offending method.

https://github.com/28mm/blast-radius/blob/528d89d9a0b283fbfa78685a541c0fdbd7eb24c4/blastradius/handlers/dot.py#L443-L448

Unfortunately I'm away from my development environment right now, so a fix will likely wait till tomorrow--I'll let you know.

SIDE NOTE: This is probably the same error you're getting in --serve mode, just without the errors suppressed. Ultimately --serve will be a better way to get the svg, because coloration and interactivity are all implemented in javascript.

[AnthonyWC]

Ok if this helps in any way, I wrap the offending code in try/except block; the next error was at:

 File "/Users/antho/Library/Python/3.6/lib/python/site-packages/blastradius/handlers/dot.py", line 422, in __init__
    self.modules = [ m for m in self.module.split('.') if m != 'module' ]
AttributeError: 'NoneType' object has no attribute 'split'

I wrap this to check if self.module is not None, then I was able to generate the graph with terraform graph | blast-radius --svg > graph.svg

Running blast-radius --serve . does display the graph but the graph isn't working other than display as it is not clickable or zoomable. Console shows these errors:

* Running on http://0.0.0.0:5000/ (Press CTRL+C to quit)
127.0.0.1 - - [28/Feb/2018 10:07:45] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [28/Feb/2018 10:07:45] "GET /robots.txt HTTP/1.1" 404 -
127.0.0.1 - - [28/Feb/2018 10:07:45] "GET /static/js/jquery.slim.min.js HTTP/1.1" 304 -
127.0.0.1 - - [28/Feb/2018 10:07:45] "GET /static/js/bootstrap.min.js HTTP/1.1" 304 -
127.0.0.1 - - [28/Feb/2018 10:07:45] "GET /static/js/fontawesome-all.min.js HTTP/1.1" 304 -
127.0.0.1 - - [28/Feb/2018 10:07:45] "GET /static/js/d3.v4.js HTTP/1.1" 304 -
127.0.0.1 - - [28/Feb/2018 10:07:45] "GET /static/js/d3-tip.js HTTP/1.1" 304 -
127.0.0.1 - - [28/Feb/2018 10:07:45] "GET /static/js/blast-radius.js HTTP/1.1" 304 -
127.0.0.1 - - [28/Feb/2018 10:07:45] "GET /static/css/bootstrap.min.css HTTP/1.1" 304 -
127.0.0.1 - - [28/Feb/2018 10:07:45] "GET /static/css/selectize.css HTTP/1.1" 304 -
127.0.0.1 - - [28/Feb/2018 10:07:45] "GET /static/css/style.css HTTP/1.1" 304 -
127.0.0.1 - - [28/Feb/2018 10:07:45] "GET /static/js/categories.js HTTP/1.1" 304 -
127.0.0.1 - - [28/Feb/2018 10:07:45] "GET /static/js/svg-pan-zoom.js HTTP/1.1" 304 -
127.0.0.1 - - [28/Feb/2018 10:07:45] "GET /static/js/selectize.js HTTP/1.1" 304 -
WARNING:root:'NoneType' object has no attribute 'groupdict'
127.0.0.1 - - [28/Feb/2018 10:07:48] "GET /graph.svg HTTP/1.1" 200 -
INFO:werkzeug:127.0.0.1 - - [28/Feb/2018 10:07:48] "GET /graph.svg HTTP/1.1" 200 -
WARNING:root:'NoneType' object has no attribute 'groupdict'
ERROR:blastradius.server.server:Exception on /graph.json [GET]
Traceback (most recent call last):
  File "/Users/antho/Library/Python/3.6/lib/python/site-packages/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/Users/antho/Library/Python/3.6/lib/python/site-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/Users/antho/Library/Python/3.6/lib/python/site-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/Users/antho/Library/Python/3.6/lib/python/site-packages/flask/_compat.py", line 33, in reraise
    raise value
  File "/Users/antho/Library/Python/3.6/lib/python/site-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/Users/antho/Library/Python/3.6/lib/python/site-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/Users/antho/Library/Python/3.6/lib/python/site-packages/blastradius/server/server.py", line 59, in graph_json
    node.definition = tf.get_def(node)
  File "/Users/antho/Library/Python/3.6/lib/python/site-packages/blastradius/handlers/terraform.py", line 56, in get_def
    if len(node.modules) > module_depth and node.modules[0] != 'root':
AttributeError: 'DotNode' object has no attribute 'modules'
127.0.0.1 - - [28/Feb/2018 10:07:50] "GET /graph.json HTTP/1.1" 500 -
INFO:werkzeug:127.0.0.1 - - [28/Feb/2018 10:07:50] "GET /graph.json HTTP/1.1" 500

[28mm]

@AnthonyWC

I wonder what the value of label is that's breaking the regex. Maybe try this and post the output?

    @staticmethod
    def _module(label):
        try:
            if not re.match(r'(\[root\]\s+)*module\..*', label):
                return 'root'
            m = re.match(r'(\[root\]\s+)*(?P<module>\S+)\.(?P<type>\S+)\.\S+', label)
            return m.groupdict()['module']
        except:
            print('LABEL: ' + label)
            raise

[AnthonyWC]

Ok I used this:

    @staticmethod
    def _module(label):
        if not re.match(r'(\[root\]\s+)*module\..*', label):
            return 'root'
        m = re.match(r'(\[root\]\s+)*(?P<module>\S+)\.(?P<type>\S+)\.\S+', label)
        if m is not None:
            return m.groupdict()['module']
        else:
            raise Exception("None: ", label)

and got this: Exception: ('None: ', '[root] module.redshift_subnet_group (removed)')

[28mm]

@AnthonyWC

So, this is the first I'm encountering (removed) modules. I just pushed a fix (be89e3f85986628ec300df33a9e57d2db8cc66c0) -- Not sure that its ultimately the right approach, but it should get your graph to render.

If it works and seems reasonable I'll bump the version # and push to pypi.

Thanks.

[AnthonyWC]

I just tested the new version and it didn't throw up any Exception whereas the old version would error out; also the graph displayed correctly.