Write a PyTest hook function to parse the AST and run our checks

[aubreypc]

The various assertion checks defined in checks.py need to be called as part of a PyTest hook function so that we can retrieve the function object being tested. Then we can use inspect.getsource to retrieve the source code of the function being tested, and pass this to ast.parse to generate an abstract syntax tree.

Using for node in ast.walk(tree), we can traverse the syntax tree and decide which checks to run based on what kind of nodes we encounter. Use isinstance(node, ast.Assert), for example, to see whether a node is an assertion statement (Consult the AST docs to learn about the different types of nodes corresponding to different Python statements).

I have written a run_compare_checks function which takes in an ast.Compare object and runs a number of checks for the quality of comparisons (we can easily define more checks in the future -- see checks.py for examples). In the hook function, while we are in our for node in ast.walk(tree) loop, node will have a node.test property when it represents an assert statement in the code, i.e. when isinstance(node, ast.Assert) is True. This node.test object is the node of the tree which the assert statement is about. It could be a comparison or some other type of statement, so we need to use isinstance to switch between different cases -- right now there is only run_compare_checks for the ast.Compares case, but we can make more functions to run other types of checks. My run_compare_checks function takes in node.test as an argument and returns a list of error messages. We will also need to figure out how to report these errors in PyTest (which would be a separate issue).

[aubreypc]

Another option would be, instead of using conditional logic within the hook function to determine which checks to run based on the type of node, we could write some kind of decorator so that the logic for determining where to run the check is in the same place as the check itself. For example, the interface could look something like:

@runs_on(ast.Compare):  # Check should run when the assertion is a comparison
def is_double_negative(left, op, right):
    # does stuff...

[aubreypc]

It might also be possible to use the type hints of a function to determine when node types it should be called for, since Python reveals these as a myFunc.__annotation__ dictionary