Improved error messages

[roll]

• fixes #86

[roll]

@Stephen-Gates Could you please take a look? I've improved some error messages (syncing some with Data Quality Spec). But still it's not exactly the same in high-level because various reasons like (some of this problems are solvable):

• fields-headers mismatch is just on error though DQS uses extra/missing-header errors
• in general not enough information for field.castRow to say on which row number is error
• unlike goodtables we handle constraints here together not one-by-one
• foreign key message requires some internal refactoring to requested message data
• etc

So here is a question how far we should improve it. Is it a real blocker for data-curator. Problem here that error messages here is a exceptional situation messages. It's not intended to be user-facing in general (e.g. app on top should catch it). Unlike to goodtables where all error messages are intended to be user-facing.

So looking forward to your comments. We still have some reserves to improve some messages more (investing more time). Just looking forward to hear what's blockers/priorities. We could handle it in this PR or open individual issue for every new requested improvement.

cc @pwalsh

[roll]

List of all data related exceptions:

• [field.js]
• The value "${value}" in column "${this.name}" is not type "${this.type}" and format "${this.format}"
• Field "${this.name}" has constraint "${name}" which is not satisfied for value "{value}"
• [schema.js]
• Row length ${row.length} doesn't match fields count ${this.fields.length}
• [table.js]
• Table headers don\'t match schema field names
• Row ${rowNumber} has unique constraint violation in column "${cache.name}"
• Foreign key "${foreignKey.fields}" violation in row ${rowNumber}

[Stephen-Gates]

@roll I will have a good look over the weekend.

[Stephen-Gates]

@roll

Ideally the row and column for each error would be reported. This would enable Data Curator to:

• highlight the cells where errors are found
• support a workflow where users validate a column at a time

If this is not possible, then returning the row in error would enable the row to be highlighted.

In terms of priority:

1. Type and Format
2. Constraint
3. Foreign Key
4. Structure

I'll add a feature request and comment separately on error message wording.

[roll]

@Stephen-Gates That's what I was thinking for. Error messages may be are not the best thing to rely on. But the lib could provide an API for error context when possible e.g:

• error.row
• error.column
• error.code
• etc

Than it could be re-used in the higher-level app (including an ability to compose a message using data quality spec). I think it could be more reliable way to achieve the same goal.

[Stephen-Gates]

Sounds like a good approach @roll

[Stephen-Gates]

@roll sorry for the delay. Assuming no other variables are available for use in the error messages, then...

Type or Format Error

The value "${value}" in column "${this.name}" is not type "${this.type}" and format "${this.format}" appears to be as close as you can get to the Type or Format DQS error message in DQS.

Constraint Error

To better match the format of most constraint error messages in DQS, I suggest Field "${this.name}" has constraint "${name}" which is not satisfied for value "{value}" is rewritten to, The value "{value}" does not conform to the "${name}" constraint for field "${this.name}"

Extra / Missing Value

There's no equivalent message in DQS. The closest is the extra/missing value error messages in DQS that compares the row length to the header row. This error appears to be comparing the row length to the number of fields in the schema.

I suggest Row length ${row.length} doesn't match fields count ${this.fields.length} is rewritten to, The row with ${row.length} columns does not match the ${this.fields.length} fields in the schema

Perhaps columns should be values?

Non-matching Header

To better match the format of most non-matching header error message in DQS, I suggest Table headers don\'t match schema field names is rewritten to, The column header names do not match the field names in the schema

Unique constraint

Row ${rowNumber} has unique constraint violation in column "${cache.name}"

edit that should be applied to DQS and here,

Row ${rowNumber} has a unique constraint violation in column "${cache.name}"

Foreign key

Not implemented in DQS (although there is a suggestion). I don't think I can improve Foreign key "${foreignKey.fields}" violation in row ${rowNumber}

I did consider:

• The foreign key "${foreignKey.fields}" in row ${rowNumber} has a foreign key violation
• The foreign key "${foreignKey.fields}" in row ${rowNumber} does not have a matching primary key value

[roll]

@Stephen-Gates Thanks. It's just a great help.

[roll]

@Stephen-Gates Please take a look. I've:

• updated error messages based on your feedback
• added error.rowNumber/columnNumber properties (if available)

[Stephen-Gates]

@roll will review this week. thanks.

[roll]

@Stephen-Gates Thanks! I've fixes the last one.