monte-language / typhon

A virtual machine for Monte.
Other
67 stars 10 forks source link

capnp: AnyPointer support #220

Open MostAwesomeDude opened 3 years ago

MostAwesomeDude commented 3 years ago

We can't really assign to AnyPointer fields in Capn Proto structures. This is a showstopper for continuing to implement CapTP over Capn Proto RPC.

A solution presumably allows us to decouple writers from schemata a bit more; we need to allow two different schemata to write to the same buffer at once in order to do this correctly. We also want to be able to read from AnyPointers, so we need some sort of casting operator.

dckc commented 3 years ago

Clue me in? Why would we want / need to assign to AnyPointer fields in Capn Proto structures?

I think I know what Capn Proto structures are. I don't recall off hand what AnyPointer is. I suspect I don't understand "assign" in this context.

washort commented 3 years ago

the capnproto RPC schema has an AnyPointer field for message payloads, so that peers can define their own schema for message contents. We need a way to switch from the "outer" readers/writers to the "inner" ones for the structures referenced via AnyPointer.

MostAwesomeDude commented 3 years ago

An AnyPointer (upstream docs) is a dynamically-typed pointer. Recall that, in Capn's encoding framework, trees and structs are passed by pointer; this is a way to reinterpret which schema is being used to decode a particular pointer.

For decoding, it's not hard to imagine that we might want to take some pointer and cast it:

def pointer := struct.getPointer()
def casted := MySchema.cast(pointer)

For encoding, things are trickier due to the current structure of objects. We will probably want to create writers as we do now, but allow for a writer to take a secondary schema and spawn a new structure; this allows for the structure to be foreign but still occupy the same buffer cleanly. Something like:

def writer := makeMainWriter()
def payload := writer.withSchema(MySchema).makeMyStruct(…)
def top := writer.makeTopStruct(=> payload)

In this hypothetical design, the payload is known to the writer to be an AnyPointer; it's opaque but fits into this typed hole, just like how Ints and Strs are adapted to Capn's typing already.