capnproto / pycapnp

Cap'n Proto serialization/RPC system - Python bindings
BSD 2-Clause "Simplified" License
459 stars 124 forks source link

Any way to parse bytes as stream? #283

Open BartWeyder opened 2 years ago

BartWeyder commented 2 years ago

Hi, I get bytes (packets) of data, this data is parsed with non-capnproto format first and then rest of data is passed to me. This data contains several different capnproto structs packed one-by-one. Is there any way to work with it as with stream? I can't use IOBytes with read_packed since fileno is required.

Is there maybe some way to get bytes number that from_bytes used? So I can chop my bytes manually.

Currently the only way I see is to replicate struct, serialize it to bytes and then get its size, but it's extremely stupid.

Writing bytes into file first looks even more stupid.

phil535 commented 8 months ago

Hi,

i would also like to create buffers for network communication. A in-memory buffer would be quite handy for that.

io.BytesIO would be my first choice to do that but the write method of DynamicStructBuilder requires the fileno method.

Here is my usecase:

import pytest
import io
import capnp

import clientpacket_capnp

def test_clientpacket():
    with io.BytesIO() as buf:
        p = clientpacket_capnp.ClientPacketPlayerMove.new_message()
        pos = p.pos
        pos.x = 1
        pos.y = 2
        pos.z = 3
        p.write(buf)  # <- E   io.UnsupportedOperation: fileno

Schema:

@0xae5b31b06588e9f6;

struct Vec3 {
  x @0 :Int16;
  y @1 :Int16;
  z @2 :Int16;
}

struct ClientPacketPlayerMove {
  pos @0 :Vec3;
}

Is there another way to do that?