Closed qcn closed 1 year ago
Thank you for the report.
I went over the diff but don't see what could possibly cause this: https://github.com/msgpack/msgpack-ruby/compare/v1.5.6...v1.6.0
Could you tell us more how these payloads are generated? Do you call Factory#dump
or Packer#pack
? Do you re-use the packer, is it used concurrently? etc.
Thanks for the response! We call MessagePack.pack
each time we pack data so it'll be using the defaults, and it is possible for it to run concurrently.
Thanks for the extra information.
One last question, do you call MessagePack.pack(object, io)
or MessagePack.pack(object)
?
What's weird is that MessagePack.pack
create a new Packer
every time, so I don't get how we could have shared state here: https://github.com/msgpack/msgpack-ruby/blob/0c1119be839b332733b13e02b4406524ee0b4499/lib/msgpack.rb#L38-L42
Thanks for the extra information.
One last question, do you call
MessagePack.pack(object, io)
orMessagePack.pack(object)
?
We call MessagePack.pack(object)
.
The repetition is from within the same msgpacked object, so my first instinct would be that the problem doesn't arise from shared state between packers, but I don't think I have enough context to say for sure.
Ok, thank you. It could be a bug in the buffer implementation, but again, quite unsure what change in 1.6.0 could have caused this.
I doubt we'll be able to debug this without a reproduction.
@qcn any update on this one? Did the bug go away with 1.7.0
as suspected in #328 ?
Seems like it was indeed the same root cause: https://github.com/msgpack/msgpack-ruby/pull/328#issuecomment-1534691313
Very happy to close that issue.
Sorry for the delay - we were in a low-traffic period and didn't want to confirm it'd been fixed until we felt like we had enough data! Thanks so much for fixing this one :)
No worries, thank you for confirming!
Since upgrading to msgpack 1.6.0 we've been seeing occasional data corruption when packing, which did not happen with 1.5.6. The behaviour is non-deterministic and we haven't been able to explicitly reproduce it yet, but we have several examples of the corrupted data. The symptom is that some bytes from later in the packed data are copied over bytes from earlier, so those bytes appear twice in the packed data (and there is data loss from the corruption).
Example, using Ruby 2.7.6 and msgpack 1.6.0:
The original Ruby object:
Usually the output from running this through
MessagePack.pack
is:We've observed the following corrupted serialization from msgpack:
In this corrupted version, the bytes from positions 8192 to the end are copied over the bytes from position 4096:
In longer examples we've observed similar overwritten bytes, but it isn't necessarily from the end of the sequence and doesn't always seem to align neatly, eg. this corrupted msgpack output where 4094 bytes are repeated exactly from positions 85945 to 90038 and then 90039 to 94132:
While we're not sure why this is happening we're hoping others may have seen similar issues or be able to shed some light.