Support missing values at field level

[Stephen-Gates]

Desired Behaviour

Support missingValues at field level - possible changes in specification. See https://github.com/frictionlessdata/specs/issues/551

• Add property to column
• Use property in validation

[Stephen-Gates]

Note this PR https://github.com/frictionlessdata/specs/pull/588 for "no missing values".

Need to think about the UI for this

• table vs column level
• explaining the difference between the required constraint and no missingValues

[Stephen-Gates]

/remind me on april 24 to write the acceptance test for:

• Set Column and Table properties
• Validate Table

[reminders[bot]]

@Stephen-Gates set a reminder for Apr 24th 2018

[Stephen-Gates]

User interface thoughts:

Use missing values ui from Table Properties on Column Properties

Update tooltips to reflect if values apply to Table or Column. Explain the Column missing values inherit from the Table values Explain that missing value characters will be interpreted as null

[Stephen-Gates]

Validate different missing values by column acceptance test done

https://app.cucumber.pro/projects/data-curator/documents/branch/develop/test/features/tools/validate-table.feature#scenario-set-different-missing-values-across-columns

Cucumber Pro

[Stephen-Gates]

@mattRedBox just realised that this isn't part of the spec yet or even a pattern. I can submit a PR for a pattern. I think you said tableschema.js already supports this - correct?

Proposal https://discuss.okfn.org/t/missing-values-per-field-pattern/6571

PR https://github.com/frictionlessdata/specs/pull/608

[ghost]

Hi @Stephen-Gates They have the variable in place. At the moment, each field/column has a reference to the table level missingValues. We don't copy through this reference yet, but I guess once frictionless head in that direction, it will make it simpler for frictionless to refactor so that the missingValue is specific to each field. We will then need to update what we do.

[ghost]

Hi @Stephen-Gates So reading https://discuss.okfn.org/t/missing-values-per-field-pattern/6571, it reads correctly to me.

Open Knowledge Forums

Missing values per field pattern

Following on from Issues #551 in the specs, I’ve decided to propose a new pattern to add to the Frictionless Data Patterns. Looking forward to your feedback 😄 Missing values per field Overview Characters representing missing values in a table can be defined for all fields in a Tabular Data Resource using the missingValues property in a Table Schema. Values that match the missingValues are treated as null. The Missing values per field pattern allows different missing values to be specifie...

[reminders[bot]]

:wave: @Stephen-Gates, on to write the acceptance test for: