near / borsh

Binary Object Representation Serializer for Hashing
https://borsh.io/
487 stars 41 forks source link

Improve performance of String and Vec<u8> #58

Closed evgenykuzyakov closed 4 years ago

evgenykuzyakov commented 4 years ago

Optimizing strings/vec reads using custom implementation for Vec<u8> with a bit of unsafe code, but without UB. Switching deserialization to use &mut &[u8] instead of BufRead. It allows us to switch to BorshError instead of std::io::Error in the future to go with no_std.

Test plan

evgenykuzyakov commented 4 years ago

Is it possible that https://doc.rust-lang.org/std/io/struct.BufReader.html implements this very functionality?

I think we can use BufReader with raw fill_buf() functionality for a lot of what I did manually with read_exact

evgenykuzyakov commented 4 years ago

Input Highlights for gas fess:

Cost of Borsh input of (s: String) where s="" is 9,080,640,736 Average diff 131,604,521 Cost of Borsh input of (v: Vec) where v=[] is 8,455,908,634 Average diff 112,708,303 Cost of Borsh input of (v: Vec) where v=[] is 8,425,057,666 Average diff 651,364,504

Full results: https://github.com/nearprotocol/nearcore/issues/1829#issuecomment-596137624

TODO: Add Vec<_> tests

MaksymZavershynskyi commented 4 years ago

Could you rerun https://github.com/nearprotocol/borsh/tree/master/borsh-rs/benchmarks and re-generate the graphs?

MaksymZavershynskyi commented 4 years ago

Could you also bump the versions?

evgenykuzyakov commented 4 years ago

Trying switch to fn deserialize(buf: &mut [u8]) -> Result<Self, Error>;. Will let you know

evgenykuzyakov commented 4 years ago

See https://github.com/nearprotocol/borsh/pull/62 for slice reader version. Need to fix benchmarks, but overall looks clean.

Also once we switch serialize to &mut Vec<u8>, we can kill std:io::Error

evgenykuzyakov commented 4 years ago

Going to regenerate benchmarks with the slice run

evgenykuzyakov commented 4 years ago

Merged buf slice branch, so it's ready to go (once the new benchmarks are added)