modify duplicate and sort processors to support latest kvfile features - Githubissues

datahq / dataflows

DataFlows is a simple, intuitive lightweight framework for building data processing flows in python.

https://dataflows.org

MIT License

194 stars 39 forks source link

modify duplicate and sort processors to support latest kvfile features #9

Closed OriHoch closed 5 years ago

OriHoch commented 5 years ago

Latest kvfile release contains the following features:

bulk insert (Significantly reduces speed difference between LevelDB and Sqlite)
reverse sort order

This PR integrates those changes to the duplicate and sort_rows processors

Added tests and modified tox to test with LevelDB and Sqlite

tox.ini installs kvfile from my modified branch, once bulk insert is merged - should remove that

coveralls commented 5 years ago

Pull Request Test Coverage Report for Build 29

5 of 5 (100.0%) changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.05%) to 73.214%

Totals
Change from base Build 25:	-0.05%
Covered Lines:	820
Relevant Lines:	1120

💛 - Coveralls

coveralls commented 5 years ago

Pull Request Test Coverage Report for Build 29

5 of 5 (100.0%) changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.05%) to 73.214%

Totals
Change from base Build 25:	-0.05%
Covered Lines:	820
Relevant Lines:	1120

💛 - Coveralls

coveralls commented 5 years ago

Pull Request Test Coverage Report for Build 31

10 of 10 (100.0%) changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.1%) to 73.166%

Totals
Change from base Build 25:	-0.1%
Covered Lines:	818
Relevant Lines:	1118

💛 - Coveralls