capnp::Data equivalent in JS

[jonathan-roy]

Hi,

I have a Request schema with a payload field of type AnyPointer. This field is sometime used as a capnp::Data struct and I could not find out what is the correct way to parse this capnp::Data struct in JS and serialize back the data to a capnp::Data struct. I tried to simply send a Buffer but I have the following error:

Error: expected array.size() >= offset; Message ends prematurely in segment table.

Can someone help please?

[kentonv]

Unfortunately I'm not sure if there is an easy answer to this currently in node-capnp. An AnyPointer field in the current implementation is represented as an encoded Cap'n Proto message. So it's a buffer, but that buffer is expected to start with a segment table followed by a root pointer which points at the root object. In a normal message, the root object would be a struct. But if the AnyPointer contains a Data value, then the root pointer in this representation is a Data pointer. There isn't a convenient API to construct such an odd message.

I guess as a hack, you could define two types:

struct AnyBox { value @0 :AnyPointer; }
struct DataBox { value @0 :Data; }

Now, you can construct a DataBox containing your data as the value. Serialize that, then take the serialized buffer and immediately parse it again as an AnyBox. Now the value field contains the AnyPointer representation of the original data buffer. You can take that value and shove it into you real message's AnyPointer field.

All this is, of course, not very efficient. :/

[jonathan-roy]

Thank you for your quick reply, the hack you propose is working :-)

Can you see a way to handle the reverse operation as well: read "application data" in JS from a capnp::Data?

[kentonv]

Yes, the reverse operation can be done by reversing the steps: serialize your AnyPointer value in an AnyBox, then parse it as a DataBox.

[jonathan-roy]

Of course! Thank you for your help!