capnproto / pycapnp

Cap'n Proto serialization/RPC system - Python bindings
BSD 2-Clause "Simplified" License
458 stars 125 forks source link

Premature EOF with multiple read() #325

Closed fungs closed 5 months ago

fungs commented 10 months ago

Hi,

this is probably more of a clarification request. I know that pycapnp wraps all the C++ code, but in case this is a bug, I wanted to have this referenced here first. It might also help other users since the documentation is scarce at the moment.

Simple case: I'm reading a sequence of serialized objects from a file stream (a fifo in my case). On the sender side, these are just produced using filestream_object_async.write(capnp_object.to_bytes()) (using async io) or capnp_object.write(filestream_object_sync) (using sync io). When I read the stream, I can do this either using

  1. capnp_object_type.read_multiple(filestream_object_sync) with iterator syntax
  2. capnp_object_type.read(filestream_object_sync) multiple times

This seems to work equivalently except the last item sent. Using read(), this produces a parsing error:

File "capnp/lib/capnp.pyx", line 3229, in capnp.lib.capnp._StructModule.read
File "capnp/lib/capnp.pyx", line 3798, in capnp.lib.capnp._StreamFdMessageReader.__init__
capnp.lib.capnp.KjException: kj/io.c++:53: failed: expected n >= minBytes [0 >= 8]; Premature EOF

It this intended behavior?

fungs commented 5 months ago

I'm closing this myself. The real issue is, that the documentation doesn't specify any behavior or return type when the file ends (when it hits EOF). This confused my back then.

This is the crude way pycapnp determines internally the end of a file stream.

except capnp.KjException as e:
  if 'EOF' in str(e):