Allow Exceptions to Bubble Up

jdrew82 commented 5 months ago

When working with the generate_config function the current behavior has any Exceptions from Jinja2 being masked by raising a NornirNautobotException with log message. This means that any logging that is done in a Job is unable to see the full traceback. If we instead re-raised the same Exception we'd have access to the traceback and could log it. The traceback is essential with config generation as it allows quick identification of where the error lies. For example:

The full traceback is this:

[ERROR/ForkPoolWorker-6] Host 'jcy-spine-01.infra.ntc.com': task 'template_file' failed with traceback:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/nornir/core/task.py", line 99, in start
    r = self.task(self, **self.params)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/nornir_jinja2/plugins/tasks/template_file.py", line 43, in template_file
    text = t.render(host=task.host, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 1301, in render
    self.environment.handle_exception()
  File "/usr/local/lib/python3.11/site-packages/jinja2/environment.py", line 936, in handle_exception
    raise rewrite_traceback_stack(source=source)
  File "/opt/nautobot/git/templates/cisco_nxos.j2", line 103, in top-level template code
    {% include './nxos/vlans.j2' %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/nautobot/git/templates/nxos/vlans.j2", line 1, in top-level template code
    {% for vlan in site["vlans"] %}
    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/jinja2/sandbox.py", line 303, in getitem
    return obj[argument]
           ~~~^^^^^^^^^^
jinja2.exceptions.UndefinedError: 'site' is undefined

but the Job logs only show E1010: There was a jinja2.exceptions.UndefinedError error: 'site' is undefined. You have to look at the STDERR logs to find the traceback. It'd be much more useful if we could log the traceback as that's where the important information is.

itdependsnetworks commented 5 months ago

I am not sure what to do, this contract with upstream is expecting there to be a NornirNautobotException raised, so there are implications. In my humble opinion, this is where this should be handled https://github.com/nautobot/nautobot/issues/2331, otherwise it is a game of whack-a-mole in which we will never win.

jdrew82 commented 5 months ago

From our discussion I attempted to just log the traceback and unfortunately the closest you can get is the Traceback object itself but that doesn't appear to contain the full traceback, just the start of it and we need the end. We can ignore this request and consider it a not gonna work.

jdrew82 commented 5 months ago

Looks like @jmpettit figured out how to get the traceback logged in #149!

nautobot / nornir-nautobot

Allow Exceptions to Bubble Up #148