dask / hdfs3

A wrapper for libhdfs3 to interact with HDFS from Python
http://hdfs3.readthedocs.io/en/latest/
BSD 3-Clause "New" or "Revised" License
136 stars 40 forks source link

Be more strict about unicode/bytes conversions #147

Closed jcrist closed 6 years ago

jcrist commented 6 years ago

Previously this library relied upon duck-typed behavior, instead of following precise schemas for the conversions between ctypes structs and dictionaries of python objects. This led to unicode decode errors, as bytestrings not representing human readable content were being decoded.

This fixes this issue in the following ways:

jcrist commented 6 years ago

cc @brendancol

martindurant commented 6 years ago

The py3 test run aborted - does it succeed for you?

jcrist commented 6 years ago

It does on the recent release of distributed, but not on distributed master. Looks like https://github.com/dask/distributed/pull/1594 broke something. cc @pitrou, @mrocklin.

pitrou commented 6 years ago

If you want to diagnose this a bit further, I suggest:

jcrist commented 6 years ago

Ok, after removing the failing dask tests, this now passes.

martindurant commented 6 years ago

+1 for the green