search

simoninns / DomesdayDuplicator

High-speed LaserDisc RF sampler
GNU General Public License v3.0
148 stars 21 forks source link

dddconv: corruption when reading from stdin #67

Closed wondras closed 5 years ago

wondras commented 5 years ago

When data is piped from cat, or from another instance of dddconv, there are errors in the output data.

Setup:

$ md5sum ralf.lds
c6caab4dd625cba9225087f1a1c67180  ralf.lds
$ dddconv -u -i ralf.lds -o ralf.raw

When reading from a file, dddconv returns the correct data:

$ dddconv -p -i ralf.raw | md5sum
c6caab4dd625cba9225087f1a1c67180  -

When data is piped from cat, a different result is returned each time:

$ cat ralf.raw | dddconv -p | md5sum
a0a37608a5aab9598398813926fdf870  -
$ cat ralf.raw | dddconv -p | md5sum
1fcf71275055d7fe6d393bb1cf7f2c3e  -
$ cat ralf.raw | dddconv -p | md5sum
07daf88db6b66d1b5556b2aae0aa80b5  -

Piping from another instance of dddconv also returns varying results:

$ dddconv -u -i ralf.lds | dddconv -p | md5sum
21fe27693dc357d4067e80e370f91411  -
$ dddconv -u -i ralf.lds | dddconv -p | md5sum
487eb19a99f51574607d33bec603097a  -
$ dddconv -u -i ralf.lds | dddconv -p | md5sum
fc4af1c96e21b09f05b02ba0e5b5df87  -

However, when piping from FLAC, the correct result is returned:

$ flac -s --sample-rate=48000 --sign=signed --channels=1 --endian=little --bps=16 --compression-level-1 ralf.raw --output-name=- | flac -d -s --force-raw-format --endian=little --sign=signed --output-name=- - | dddconv -p | md5sum
ralf.raw: WARNING, cannot write back seekpoints when encoding to stdout
-: WARNING, cannot check MD5 signature since it was unset in the STREAMINFO
c6caab4dd625cba9225087f1a1c67180  -

Reproduces on macOS 10.14 and Ubuntu 18.0.1, in both --pack and --unpack modes.

Further troubleshooting and possible workaround:

In Linux, by using the buffer command with its maximum -s size parameter (512K), the correct md5sum is returned:

$ cat ralf.raw | buffer -s 512K | dddconv -p | md5sum
c6caab4dd625cba9225087f1a1c67180  -

Smaller buffer sizes still have incorrect results:

$ cat ralf.raw | buffer -s 256K | dddconv -p | md5sum
27b4424865fa9f5e2504b791c47df6b8  -
simoninns commented 5 years ago

I've modified the input file buffer handling to correct the issue (which occurs when the pipe provides less than a buffer full of data). Please retest.

simoninns commented 5 years ago

Tested, fixed.