datahq / dataflows

DataFlows is a simple, intuitive lightweight framework for building data processing flows in python.
https://dataflows.org
MIT License
194 stars 39 forks source link

Fixed saving hash/bytes to a resource #118

Closed roll closed 4 years ago

roll commented 4 years ago
roll commented 4 years ago

@akariv Please take a look.

It seems to have fixed the resource.hash/bytes problem but I'm still not sure what's going on with package.hash/bytes (and what it is actually). I've added a few todos regarding it

coveralls commented 4 years ago

Pull Request Test Coverage Report for Build 411


Changes Missing Coverage Covered Lines Changed/Added Lines %
dataflows/processors/dumpers/file_dumper.py 6 7 85.71%
<!-- Total: 6 7 85.71% -->
Files with Coverage Reduction New Missed Lines %
dataflows/processors/dumpers/file_formats.py 7 79.63%
<!-- Total: 7 -->
Totals Coverage Status
Change from base Build 405: 0.04%
Covered Lines: 1662
Relevant Lines: 1962

💛 - Coveralls
akariv commented 4 years ago

Bytes is for counting the number of bytes (approx.) the package weighs. Hash is for saving the file in a hashed path (that is, re-saving the package in the same location btu with different data won't override the previous data)

roll commented 4 years ago

@akariv I didn't get that inc stands for increment in the first place.

I've removed the todos. It seems it works correctly.

The PR fixes saving byte count to resources.

cschloer commented 4 years ago

Just bumping this issue! I know its holiday season so no real rush but just want to keep this on our radar.

roll commented 4 years ago

I hope it's ready for a review