search

jerry73204 / rust-tfrecord

A Rust crate that reads and writes tfrecord files
MIT License
25 stars 8 forks source link

BufReader Checksum Error #3

Closed Leoyzen closed 3 years ago

Leoyzen commented 3 years ago

Hello, thanks for your contribution. I encountered problem when using this repo try reading tfrecord with check_integrity=true.

[2021-06-02 13:52:20][ERROR]idx@62680: UnexpectedEofError
[2021-06-02 13:52:20][ERROR]idx@62681: ChecksumMismatchError { expect: "0x0a75f6ec", found: "0x63c4f097" }
[2021-06-02 13:52:20][ERROR]idx@62682: ChecksumMismatchError { expect: "0x0a0b1263", found: "0x71959e9e" }
[2021-06-02 13:52:20][ERROR]idx@62683: ChecksumMismatchError { expect: "0x1b0a6572", found: "0x801e5dc9" }
[2021-06-02 13:52:20][ERROR]idx@62684: ChecksumMismatchError { expect: "0xd3100a12", found: "0x92913227" }
[2021-06-02 13:52:20][ERROR]idx@62685: ChecksumMismatchError { expect: "0x4635efaf", found: "0xcaa9b1d1" }
[2021-06-02 13:52:20][ERROR]idx@62686: ChecksumMismatchError { expect: "0x67616d69", found: "0x690cd862" }                           
[2021-06-02 13:52:20][ERROR]idx@62687: ChecksumMismatchError { expect: "0xd8ff27a2", found: "0x16c5f22f" }

It seems reading len_buf failed when using BufReader and meet the incompetible end of buffer. After some digging, I found this: https://github.com/rust-lang/rust/issues/22570 So we should change reader.read to reader.read_exact?

jerry73204 commented 3 years ago

With regard to #4, I made a fix in recent commit (link). Can you check if it works for you?

jerry73204 commented 3 years ago

The issue is fixed and release 0.8.0 is now available. Closing it.