frictionlessdata / tableschema-py

A Python library for working with Table Schema.
https://frictionlessdata.io
MIT License
260 stars 41 forks source link

Add preserve_missing_values parameter to cast_value function #260

Closed cschloer closed 4 years ago

cschloer commented 4 years ago

Overview

Adding a preserve_missing_values parameter to the cast_value function so that we can preserve the value saved as a missing value while still properly casting to the type. This is useful for situations where the identifier used for the missing value has important information that needs to preserved in the final data set, but shouldn't get in the way of properly casting the type of the field. For example, n/a might mean that no data was recorded, but another missing_value identifier might mean that the measured amount was below the threshold of the instrument's ability to measure.

This requires changes to dataflows and datapackage_pipelines to make sure it works with the dump_to_path and set_types processors. Those PRs are here: https://github.com/frictionlessdata/datapackage-pipelines/pull/175 https://github.com/datahq/dataflows/pull/119


Please preserve this line to notify @roll (lead of this repository)