Use sourceFileRange from conduit-extra in Control.Distributed.Dataset.LocalTmpShuffleStore.sourceFile

utdemir / distributed-dataset

A distributed data processing framework in Haskell.

BSD 3-Clause "New" or "Revised" License

114 stars 5 forks source link

Closed axman6 closed 5 years ago

axman6 commented 5 years ago

Using this should avoid the the potentially error prone arithmetic and edge cases involved with handling offsets by hand - http://hackage.haskell.org/package/conduit-extra-1.3.1.1/docs/Data-Conduit-Binary.html#v:sourceHandleRange.

utdemir commented 5 years ago

Looks much nicer now, thanks!