zakarumych / alkahest

Fantastic serialization library
Other
157 stars 9 forks source link

Add size_hint for Bincoded #16

Closed vlovich closed 1 year ago

vlovich commented 1 year ago

Also fixes issue #14

zakarumych commented 1 year ago

size_hint is intended to be extremely fast. It is used internally for optimizing out one memmove If it can't it should return None. Traversing value using serde is antonym for fast TBH :)

vlovich commented 1 year ago

Hmm... so I'm trying to allocate an I/O buffer up-front (not a Vec, not resizable) that will contain the serialized output. The serialized type has a Bincoded type. How do I figure out the size of the I/O buffer to allocate?

zakarumych commented 1 year ago

Using this

zakarumych commented 1 year ago

In general, when doing I/O you probably want to allocate buffer once and reuse it each time you send a message. In this case you can do the following: Allocate buffer that would probably fit a message of expected size. Try to serialize it optimistically. If buffer is too small - grow. Either find out exact size needed or double buffer size until message fits. Or use hybrid approach.

vlovich commented 1 year ago

I'm using glommio so there's not really any opportunity to manually reuse buffers - it has its own pool. Nor would it make sense to hang onto them for my use-case.

zakarumych commented 1 year ago

It's actually the same thing. Except that lib is keeping the buffer. Ask it for a buffer. Ask for a bigger one if message doesn't fit.

vlovich commented 1 year ago

So I'm still struggling to get Bincoded / SerializeRef working properly. Here's my code snippet:

Fails with this error

If I omit the SerializeRef derivation, it fails with:

error[E0277]: the trait bound `Struct: SerializeRef<StructFormula>` is not satisfied
   --> src/main.rs:42:52
    |
42  |     alkahest::serialized_size::<StructFormula, _>(&foo);
    |     ---------------------------------------------  ^^^ the trait `SerializeRef<StructFormula>` is not implemented for `Struct`
    |     |
    |     required by a bound introduced by this call
    |
    = note: required for `&Struct` to implement `alkahest::Serialize<StructFormula>`
note: required by a bound in `serialized_size`
   --> ~/.cargo/registry/src/index.crates.io-6f17d22bba15001f/alkahest-0.3.0/src/serialize.rs:441:8
    |
438 | pub fn serialized_size<F, T>(value: T) -> (usize, usize)
    |        --------------- required by a bound in this function
...
441 |     T: Serialize<F>,
    |        ^^^^^^^^^^^^ required by this bound in `serialized_size`
help: consider dereferencing here
    |
42  |     alkahest::serialized_size::<StructFormula, _>(&*foo);
    |                                                    +

For more information about this error, try `rustc --explain E0277`.
zakarumych commented 1 year ago

17 should help

vlovich commented 1 year ago

Thanks!