RussBaz / enforce

Python 3.5+ runtime type checking for integration testing and data validation
542 stars 21 forks source link

Add Recursion Limit to Parser #7

Open TheDataLeek opened 8 years ago

TheDataLeek commented 8 years ago

Currently, if one were to enforce typing on a function that took a list as input, and the list was especially nested and/or large, it would iterate/recurse over every object asserting that the typing matches.

For example:

import enforce
@enforce.runtime_validation
def foo(bar: List[int]) -> int:
    return bar[0]
foo(range(10000000))

An easy workaround (currently) is to just define the input type as Any, which prevents further iteration

import enforce
@enforce.runtime_validation
def foo(bar: Any) -> int:
    return bar[0]
foo(range(10000000))

This has the downside of losing the type annotations. I propose the following syntax:

import enforce
@enforce.runtime_validation(recursion_limit=0)   # not necessarily the name
def foo(bar: List[int]) -> int:
    return bar[0]
foo(range(10000000))

Setting the recursion_limit to zero would mean that just the top level nodes are parsed, while one would mean that the top level nodes and each of their children would be parsed, etc. In the above example, this would mean that we would enforce the type of bar as a List, but we wouldn't check the types of its elements (basically turning it from List[int] to List[Any]).

This change would have huge benefits for time and space complexity, as currently large arrays are completely parsed.

This change would also require a minor rework of the core decorator in order to use arguments.

RussBaz commented 8 years ago

I have removed these options from the code for now as they were not doing anything yet. We will need to reassess them when we will be dealing with the lazy evaluation of the lists and other container types.