search

dask / hdfs3

A wrapper for libhdfs3 to interact with HDFS from Python
http://hdfs3.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
136 stars 40 forks source link

read_block should include terminator #88

Closed martindurant closed 7 years ago

martindurant commented 8 years ago

We chop off the end-of-block delimiter when doing read_block, but should not.

mrocklin commented 8 years ago

Why or why not? I have a vague recollection that this was causing trouble for someone at some point?

martindurant commented 8 years ago

It fits with things like readline[s]() to keep it. We changed it in s3fs after discussion, but not here. Seeing this now, because I'm adapting these tests for client.

mrocklin commented 8 years ago

Alright. Makes sense to me, especially if other tools agree and usage dictates.