Add load infer, cast and error strategies - Githubissues

datahq / dataflows

DataFlows is a simple, intuitive lightweight framework for building data processing flows in python.

https://dataflows.org

MIT License

195 stars 40 forks source link

Add load infer, cast and error strategies #88

Closed akariv closed 5 years ago

akariv commented 5 years ago

This PR generalizes the way that load infers and casts data types, deprecating the validate and force_strings arguments into three 'strategy' arguments: infer_strategy, cast_strategy and on_error (to decide what to do when validations errors occur).

See here for usage: https://github.com/datahq/dataflows/pull/88/files#diff-0edecdf275b79739bdb4ca088cfb67e5R67

coveralls commented 5 years ago

Pull Request Test Coverage Report for Build 304

35 of 40 (87.5%) changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.01%) to 83.944%

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
dataflows/processors/load.py	34	39	87.18%
<!--	Total:	35	40	87.5%	-->

Totals
Change from base Build 303:	-0.01%
Covered Lines:	1558
Relevant Lines:	1856

💛 - Coveralls

coveralls commented 5 years ago

Pull Request Test Coverage Report for Build 309

42 of 42 (100.0%) changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.3%) to 84.222%

Totals
Change from base Build 303:	0.3%
Covered Lines:	1564
Relevant Lines:	1857

💛 - Coveralls

rufuspollock commented 5 years ago

@akariv what issue does this relate to? What is it adding? Wasn't super clearr.

akariv commented 5 years ago

@rufuspollock updated description of the PR, thanks :) !