frictionlessdata / goodtables.io

Data validation as a service. Project retired, got to the current one at frictionsless/repository
https://goodtables.io
GNU Affero General Public License v3.0
69 stars 16 forks source link

Internal server error on pattern constraint #313

Closed afuetterer closed 4 years ago

afuetterer commented 6 years ago

I try to setup a pattern constraint on a string field to match only lowercase characters and a hypen "^([-a-z])+$". If the strings are valid I get a "Valid Table" batch. But if I add some upper case characters in the rows I get internal error instead of invalid.

My field looks like this: { "name": "study", "type": "string", "constraints": { "pattern": "^([-a-z])+$" } }

Am i missing something?

amercader commented 6 years ago

@afuetterer you are not, that's a bug on our side. Thanks for the report.

We are not catching the CastError thrown by tableschema when the pattern check doesn't pass:

CastError: Field "study_name" has constraint "pattern" which is not satisfied for value "SOEP-core"
  File "celery/app/trace.py", line 368, in trace_task
    R = retval = fun(*args, **kwargs)
  File "celery/app/trace.py", line 623, in __protected_call__
    return self.run(*args, **kwargs)
  File "goodtablesio/tasks/validate.py", line 51, in validate
    report = inspector.inspect(validation_conf['source'], preset='nested')
  File "goodtables/inspector.py", line 80, in inspect
    table_warnings, table_report = task.get()
  File "multiprocessing/pool.py", line 644, in get
    raise self._value
  File "multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "goodtables/inspector.py", line 214, in __inspect_table
    check_func(errors, cells, row_number)
  File "goodtables/checks/required_constraint.py", line 25, in required_constraint
    valid = valid and cell['field'].cast_value(cell['value']) is not None
  File "tableschema/field.py", line 100, in cast_value
    ).format(field=self, name=name, value=value))

cc @roll

afuetterer commented 6 years ago

@amercader thank you for the reply. So the regex actually works. I was beginning to doubt that it was a valid expression.