search

frictionlessdata / data-quality-spec

A spec for reporting errors in data quality.
MIT License
20 stars 3 forks source link

Updated spec, schema based on goodtables.next #6

Closed roll closed 7 years ago

roll commented 7 years ago

Here is a question - should be weight hard-coded to the spec or it's more application/user level thing (cc @pwalsh @amercader)

pwalsh commented 7 years ago

@roll yes, hard code it. We don't use it in GT anyway right now.

pwalsh commented 7 years ago

@roll you've completely dropped name and message. Why?

roll commented 7 years ago

@pwalsh Thanks.

name

my bad. will add

weight

OK. But I have a question - why numbers like 9, 6 etc Not 5, 10 etc Just curious)

general vs format

I was just trying to find other word then format because there is already format-error (just one of the type). Also errors like http-error don't fit very well with format and here could be added other general error we don't know for now. But it's your call. I'm OK with any.

array vs dict

Dict is better for users but GT requires ordered data structure. Let's handle order on GT level having dict in spec. Anyway using spec literally could be a problem. So it could be better to have some middleware structure on Inspector level (to provide order e.g.).

context

Yes, it's vital for GT but it could be managed on GT level. Other question that in my opinion it's more than just implementation detail. Context (WHERE error is) seems pretty natural to describe tabular data:

dataset

But it's ok to remove from spec.

message and description

This one is the most questionable for me. Hard-coding message templates will couple spec and implementations too much in my opinion. E.g. I've added many new data to messages for GT.next like duplicate-row's additionally row numbers with the same values etc I see it as an active ongoing process (improving this messages) further. And if we're going to make all changes thru spec it could slow down this improvements. I suppose better to have something like recommended message or just a good description and let implementation handle concrete messages. WDYT?

pwalsh commented 7 years ago

@roll is goodtables now using these messages? Apart from that, all looks good.

roll commented 7 years ago

@pwalsh I'll next make a separate PR to update GT to work with this spec (and will use this messages as templates). So it's the last thing.