marl / jams

A JSON Annotated Music Specification for Reproducible MIR Research
ISC License
186 stars 27 forks source link

Validation error should be more informative #38

Closed urinieto closed 9 years ago

urinieto commented 9 years ago

A couple of weeks ago, every time I tried to validate a non-valid JAMS file I got an error reporting the exact source of the problem.

However, now I only get this:

...
  File "/home/uri/Projects/jams/jams/pyjams.py", line 1091, in save
    self.validate(strict=strict)
  File "/home/uri/Projects/jams/jams/pyjams.py", line 1124, in validate
    valid &= ann.validate(strict=strict)
  File "/home/uri/Projects/jams/jams/pyjams.py", line 781, in validate
    six.raise_from(SchemaError, invalid)
  File "/usr/local/lib/python2.7/dist-packages/six-1.9.0-py2.7.egg/six.py", line 692, in raise_from
    raise value
jams.exceptions.SchemaError

Is it possible to print out the SchemaError message by default?

urinieto commented 9 years ago

Oh, the error message of SchemaError is empty, I guess that's why nothing is printed. We should change that.

bmcfee commented 9 years ago

Oh, the error message of SchemaError is empty, I guess that's why nothing is printed.

That should never happen. Can you post a complete example that causes this?

A couple of weeks ago, every time I tried to validate a non-valid JAMS file I got an error reporting the exact source of the problem.

Nothing should have changed, except for the exception type; the same message should propagate through.

urinieto commented 9 years ago

Here you have the jams file that causes this.

bmcfee commented 9 years ago

Interesting. On python 3 (where raise_from is well-defined), it works as expected:

In [2]: J = jams.load('/home/bmcfee/Desktop/bad_jams.jams')
---------------------------------------------------------------------------
ValidationError                           Traceback (most recent call last)
/home/bmcfee/git/jams/jams/pyjams.py in validate(self, strict)
    776             for rec in records:
--> 777                 jsonschema.validate(rec, schema)
    778 

/home/bmcfee/miniconda/envs/test_env/lib/python3.4/site-packages/jsonschema/validators.py in validate(instance, schema, cls, *args, **kwargs)
    427     cls.check_schema(schema)
--> 428     cls(schema, *args, **kwargs).validate(instance)

/home/bmcfee/miniconda/envs/test_env/lib/python3.4/site-packages/jsonschema/validators.py in validate(self, *args, **kwargs)
    116             for error in self.iter_errors(*args, **kwargs):
--> 117                 raise error
    118 

ValidationError: 'C#:modal' does not match '^N|([A-G][b#]?)(:(major|minor|ionian|dorian|phrygian|lydian|mixolydian|aeolian|locrian))?$'

Failed validating 'pattern' in schema['properties']['value']:
    {'pattern': '^N|([A-G][b#]?)(:(major|minor|ionian|dorian|phrygian|lydian|mixolydian|aeolian|locrian))?$',
     'type': 'string'}

On instance['value']:
    'C#:modal'

The above exception was the direct cause of the following exception:

SchemaError                               Traceback (most recent call last)
<ipython-input-2-927dac142e95> in <module>()
----> 1 J = jams.load('/home/bmcfee/Desktop/bad_jams.jams')

/home/bmcfee/git/jams/jams/pyjams.py in load(filepath, validate, strict)
    115 
    116     if validate:
--> 117         jam.validate(strict=strict)
    118 
    119     return jam

/home/bmcfee/git/jams/jams/pyjams.py in validate(self, strict)
   1122 
   1123         for ann in self.annotations:
-> 1124             valid &= ann.validate(strict=strict)
   1125 
   1126         return valid

/home/bmcfee/git/jams/jams/pyjams.py in validate(self, strict)
    779         except jsonschema.ValidationError as invalid:
    780             if strict:
--> 781                 six.raise_from(SchemaError, invalid)
    782             else:
    783                 warnings.warn(str(invalid))

/home/bmcfee/miniconda/envs/test_env/lib/python3.4/site-packages/six.py in raise_from(value, from_value)

On python2, only the SchemaError exception makes it through:

In [2]: J = jams.load('bad_jams.jams')
---------------------------------------------------------------------------
SchemaError                               Traceback (most recent call last)
<ipython-input-2-3e7d9e363c07> in <module>()
----> 1 J = jams.load('bad_jams.jams')

/home/bmcfee/git/jams/jams/pyjams.pyc in load(filepath, validate, strict)
    115 
    116     if validate:
--> 117         jam.validate(strict=strict)
    118 
    119     return jam

/home/bmcfee/git/jams/jams/pyjams.pyc in validate(self, strict)
   1122 
   1123         for ann in self.annotations:
-> 1124             valid &= ann.validate(strict=strict)
   1125 
   1126         return valid

/home/bmcfee/git/jams/jams/pyjams.pyc in validate(self, strict)
    779         except jsonschema.ValidationError as invalid:
    780             if strict:
--> 781                 six.raise_from(SchemaError, invalid)
    782             else:
    783                 warnings.warn(str(invalid))

/usr/local/lib/python2.7/dist-packages/six.pyc in raise_from(value, from_value)
    690 else:
    691     def raise_from(value, from_value):
--> 692         raise value
    693 
    694 

SchemaError: 

But if you run it with strict=False on py2, the warning shows up as expected:

In [3]: J = jams.load('bad_jams.jams', strict=False)
/home/bmcfee/git/jams/jams/pyjams.py:783: UserWarning: u'C#:modal' does not match u'^N|([A-G][b#]?)(:(major|minor|ionian|dorian|phrygian|lydian|mixolydian|aeolian|locrian))?$'

Failed validating u'pattern' in schema[u'properties'][u'value']:
    {u'pattern': u'^N|([A-G][b#]?)(:(major|minor|ionian|dorian|phrygian|lydian|mixolydian|aeolian|locrian))?$',
     u'type': u'string'}

On instance[u'value']:
    u'C#:modal'
  warnings.warn(str(invalid))

I'll investigate.

urinieto commented 9 years ago

Thanks titan.

bmcfee commented 9 years ago

f284a75 ought to fix this

urinieto commented 9 years ago

Yay! Thanks!