Closed thenameless314159 closed 1 year ago
I forgot to say, the purpose of this library is to reduce or remove completely the framing/serialization logic contained within protocol-specific message records/POCOs.
The best would be to use auto-generated serializer via ET or the converter pattern used on the System.Text.Json (i've made a serialization library that provides this kind of stuff (not impletement in the public repo currently)) to convert messages into payload.
This would make dispatching messages easier, let's say you have some kind of IFrameDispatcher with a decorator pattern to register the messages handlers like this (assuming your protocol contains a message id in the frame metadata) :
public class FrameDispatcher : IFrameDispatcher, IFrameDispatcherBuilder
{
public FrameDispatcher(IDeserializer deserializer, IMessageHandlerFactory factory) =>
(_deserializer, _handlerFactory) = (deserializer, factory);
private delegate ValueTask Handler(Frame frame, IFrameEncoder encoder);
private readonly Dictionary<int, Handler> _handlers = new();
private readonly IMessageHandlerFactory _handlerFactory;
private readonly IDeserializer _deserializer;
public ValueTask OnFrameReceivedAsync(in Frame frame, IFrameEncoder encoder) =>
!_handlers.TryGetValue(frame.Metadata.MessageId, out var handler)
? // throw or do some handling to notify client message is not handled
: handler(frame, encoder);
public void Map<TMessage>(int withId) where TMessage : new()
{
_handlers[withId] = onFrame;
async ValueTask onFrame(Frame frame, IFrameEncoder encoder)
{
var payload = frame.Payload;
var message = !payload.IsEmpty()
? _deserializer.Deserialize<TMessage>(ref payload)
: new TMessage();
// couldn't deserialize
if(message is null) throw new InvalidDataException();
var handler = _handlerFactory.Get();
await foreach(var response in handler.OnMessageReceived(context.ConnectionClosed))
await encoder.WriteAsync(in response).ConfigureAwait(false);
// or just await encoder.WriteAsync(handler.OnMessageReceived(), context.ConnectionClosed);
}
}
}
And for the message encoding you could make an extension of the PipeFrameEncoder that would take an ISerializer and expose a ValueTask WriteMessageAsync<TMessage>(TMessage message, CancellationToken = token)
method.
I took a quick look and it does look quite promising. I'll take a deeper look and provide more detailed feedback.
While I'm exited on the topic of having a better approach defining protocols, I was holding off on some work until it was better defined with Bedrock. @thenameless314159 would this approach work to convert from objects > bytes (and vice-versa) if we had defined classes around something like Modbus? See link for example https://ipc2u.com/articles/knowledge-base/detailed-description-of-the-modbus-tcp-protocol-with-command-examples/#desc. If so, I can begin to really see the value of this (for client/server scenarios) and how it plugs into middleware pipelines.
@shaggygi This approach is not about converting objects to payload, I made a serialization library for this purpose.
Here you only have frame metadata parsing logic, the payload to object convert logic is up to you, whether it's with an implementation of my library or something else (json serialization, payload-less frames with metadata that provides infos to find the relevant message from a different channel...).
I think that I already answered all your questions on my previous comment so you may have to re-read them. I didn't look your protocol spec in depth but it seems that it's a length prefixed protocol so my framing lib or the message reader/writer of @davidfowl would be a perfect match to implement it. There are already some implementations of the message reader/writer on this repository to help you build your own. Here is the rabbitMQ protocol implementation : https://github.com/davidfowl/BedrockFramework/tree/master/src/Bedrock.Framework.Experimental/Protocols/RabbitMQ
@thenameless314159 have you tried to migrate the protocols in here to use your new APIs (especially the text based protocols)?
No I haven't yet because the framing library was made to handle a specific game protocol at first. I might have been biased while writing it then? Because on my current (private) projects using this library, the goal was to respect the SRP principle and only having to define POCOs/records for the network messages (since they're automatically generated from the game client protocol using custom tools) without any serialization or framing logic/methods in them. Therefore, it might not be well adapted to text based protocols if they're not length prefixed protocols, for instance with separators instead. I assume you wouldn't need a framing library if you have basic text-based protocols that use a separator? I believe, for this purpose, a simple class over the PipeReader should be sufficient, correct ?
However, I can try to implement the rabbitmq protocol, it should fit perfectly. What do you think, should I try to implement text based protocol first to adapt the framing library to fit both or would you want to see a length prefixed protocol implementation first?
EDIT: I started to implement the sample applications and a custom length prefixed protocol using the provided framing and serialization APIs
I finally implemented some samples in the repository, a length-prefixed text-based protocol like your echo server but using the framing APIs, and an id prefixed binary-based protocol using the serialization and framing APIs. I also provided a second sample of the id prefixed protocol using the frame dispatcher mechanism I talked about, but with a simplified and straightforward implementation of it with no lookup/auto registration/method discovery logic atm (the aim would be to have something similar as your SignalR library with the lookup/descriptors and method discovery)
Hello, first of all thank you very much for this library, I learned a lot reading the wonderful code that it contains.
I have a proposal to make for a new protocol abstraction that is currently published on this repository. I believe it can make implementation of any kind of length prefixed protocol less painful and easier in a different way than with the
IMessageReader
andIMessageWriter
protocol abstractions of @davidfowl.The logic is contained within the Andromeda.Framing library which provide read/write mechanism to handle any kind of length prefixed protocol.
Both mechanism works around the
Frame
andFrame<TMetadata>
readonly structs. Here is a less verbose version of the frames (without docs) :How it works
The library provides abstractions that must be implemented for a any kind of protocol such as
IFrameMetadata
,IMetadataEncoder
,IMetadataDecoder
, and anIMetadataParser
which inherit from both two previous interfaces.Here is the
MetadataParser<TMetadata>
base abstraction to implement :Once you've a protocol-specific implementation of an
IMetadataParser
you can use the main mechanism provided by theIFrameEncoder
and theIFrameDecoder
interfaces.The first mechanism is implemented by the
PipeFrameEncoder
class which can be thread synchronizeded (or not) to write frames in aPipeWriter
orStream
. A typed implementation also exists to handle typedFrame<TMetadata>
.The second one is implemented by the
PipeFrameDecoder
class which provides methods to read single frames or consume them via anIAsyncEnumerable<Frame>
. No thread synchronization is provided since read are mostly done with loops. A typed implementation also exists to handle typedFrame<TMetadata>
.Here is a pseudo-code sample use using these APIs with untyped decoder/encoder :
I unit tested and documented most of the code, all the tests can be found on the repository.
Please let me know what you think of my protocols APIs I would really appreciate any kind of review. Of course i'm still learning so it might contains some bad code, and I didn't benched nor profiled the performance of the whole library so it's still a todo.