RussBaz / enforce

Python 3.5+ runtime type checking for integration testing and data validation
542 stars 21 forks source link

Incorrect Exception Message #32

Closed kenfar closed 7 years ago

kenfar commented 7 years ago

Cool project. Here's a minor issue I ran into:

E enforce.exceptions.RuntimeTypeError: E The following runtime type errors were encountered: E Return value was not of type typing.Tuple[str, str, datetime.datetime]. Actual type was <class 'tuple'>

Which is misleading since the issue wasn't with the tuple type, but with the contents of the tuple. The actual message should have been something like: "Return value was not of type Tuple[str, str, dt]. Actual type was Tuple[str, int, dt].

Code that ran into this issue:

@enforce.runtime_validation
def parse_fn(fn: str) -> Tuple[str, str, dt]:
    # feed-foo_cust-acme_sen-290_sver-1_letime-20161201T041042_uuid-243c5a95d14f449dbc9b4e62b8a241d9.csv.gz
    fn = basename(fn)
    try:
        kvparts  = fn.split('.')[0].split('_')
        cust_name = kvparts[1].split('-')[1]
        sensor_id = int(kvparts[2].split('-')[1])
        insert_dt = dt.strptime(kvparts[4].split('-')[1], '%Y%m%dT%H%M%S')
    except IndexError:
        raise ValueError('invalid filename: {}'.format(fn))
    else:
        return cust_name, sensor_id, insert_dt
TheDataLeek commented 7 years ago

So I can reproduce this with an easier-to-parse snippet:

#!/usr/bin/env python3.5

from typing import Tuple
from enforce.decorators import runtime_validation

@runtime_validation
def test_func(x: str) -> Tuple[str, int, str]:
    return x, int(x), int(x)

test_func('5')

With stack trace:

Traceback (most recent call last):
  File "./tmp.py", line 11, in <module>
    test_func('5')
  File "/home/william/.local/lib/python3.5/site-packages/enforce/decorators.py", line 123, in universal
    return enforcer.validate_outputs(result)
  File "/home/william/.local/lib/python3.5/site-packages/enforce/enforcers.py", line 97, in validate_outputs
    raise RuntimeTypeError(exception_text)
enforce.exceptions.RuntimeTypeError: 
  The following runtime type errors were encountered:
        Return value was not of type typing.Tuple[str, int, str]. Actual type was <class 'tuple'>

This looks like it's working as intended, except we're telling the user the wrong info: instead of informing about the misbehaving child of an iterable we're informing about the iterable itself.

This may be desired behavior, as if the iterable is huge and we inform about the last child, it's a huge dump in a stack trace. We may instead (in this case) wish to just tell the user something like Return Value (or iterable contents) did not match type hints...

RussBaz commented 7 years ago

I have researched how exactly it works now and found a way to improve readability in some cases by including contents of the containers in the exception message. However, it might be removed when containers are finally type checking container types lazily.

It is now implemented in the dev branch. If you can, please have a look at it. (Requires Python 3.5.3+ due to a bug).

I will keep this issue opened for everyone to report problems with exception messages till the next release.

Thanks!

TheDataLeek commented 7 years ago

Yep, I'll dig in. I've been horribly busy with work the last few months, so I haven't had the time to dig into this. I'm planning on taking a look into this after Tuesday (if that's not too late).

RussBaz commented 7 years ago

I decided to release a newer version earlier. I will merge any other required fixes into 0.3.3 instead.

Please have a look at the exception message correctness when you get some time.

TheDataLeek commented 7 years ago

Sorry for delay, the new error messages look great. I think we can close this issue.