Open isikkema opened 3 months ago
Hey @isikkema, is this specifically with alloc
/std
Vec?
I think your suggestions are reasonable, though there may be some edge cases (vec of ZSTs? which is silly but possible).
I'd be open to a PR/proposal that:
deser
flavorIt's likely this wouldn't happen before #128, so I could also consider a more invasive/breaking change if you feel strongly about that.
Yes, this is with alloc Vec. Heapless Vec doesn't attempt this optimization.
Vecs of ZSTs aren't affected because it ends up pre-allocating 0 bytes regardless, but I'll add in some tests anyways.
The optimization code is in the serde crate itself. The only way to go any further here is to either know the size of the remaining byte message in the serde crate context or to know the Vec element type in the SeqAccess
or Deserializer
contexts. All in all, I think this is the best solution.
If the size portion of a serialized Vec gets corrupted, it can lead to postcard expecting the size to be much much larger than it actually is.
Example on Teensy 4.0:
Serde ends up preventing panics on most devices because it limits the amount of pre-allocated bytes to 1,048,576, but this is still too large for many low-memory embedded devices.
I propose adding the following function to
Flavor
:and using it in
SeqAccess
like so:This removes the pre-alloc optimization in cases where postcard expects more elements than there are bytes in the remaining message. This will likely result in reduced performance when the de-optimization condition is met except in the case of zero-sized types. However, it will reduce the likelihood of allocation panics, instead returning
Err(Error::DeserializeUnexpectedEnd)
.