fix the deserialization of length and keys

Lyken17 / Efficient-PyTorch

My best practice of training large dataset using PyTorch.

1.09k stars 138 forks source link

fix the deserialization of length and keys #8

Closed tahaemara closed 4 years ago

tahaemara commented 5 years ago

There an error in the old version of deserialization of length and keys because pyarrow was used to serialize both of length and keys, and the length was not deserialized besdies the keys were deserialized with msgpack not pyarrow.