search

dotnet / dotNext

Next generation API for .NET
https://dotnet.github.io/dotNext/
MIT License
1.62k stars 121 forks source link

Cluster consensus on more complex objects #39

Closed Luk164 closed 3 years ago

Luk164 commented 3 years ago

I wanted to try and use the cluster library for a project, using the provided playground as an example, but I hit a roadblock.

Multiple methods in the example have type of content that can be used for consensus limited to unmanaged types. I wish to send a more complex object, or at least a string. What am I missing? And if I am not asking for too much, can a minimal example be provided?

Thank you

Luk164 commented 3 years ago

Update: I managed to get it working after a lot of reverse engineering the example. My one complaint is that the playground is overcomplicated where it is not that helpful (multiple approaches in one example) and too simple where it matters (only sending a single float value). I would suggest creating an object, serializing and deserializing it to show the exact way to correctly handle this and maybe create a default implementation for string so JSON/XML serialization could be used out of the box.

natilivni commented 3 years ago

@Luk164, take a look at discussion #30. You can use strings for log entries.

sakno commented 3 years ago

It better to look at my answer in #40 . Closing issue.

Luk164 commented 3 years ago

@sakno Ok, I did manage to finally get it working by starting with the example playground and slowly modifying it for my purpose. You have truly done a great job with this project, though the naming scheme is a bit confusing to me sometimes.

sakno commented 3 years ago

@Luk164 , could you please provide an example of "unclear" naming?

Luk164 commented 3 years ago

@Luk164 , could you please provide an example of "unclear" naming?

I think the Write and Read functions used for serialisation and deserialization threw me off the most.

sakno commented 3 years ago

@Luk164 , because they are actually not for serialization/deserialization. They are for encoding/decoding to/from binary representation. IAsyncBinaryWriter and IAsyncBinaryReader reused for Raft-specific purposes. These interfaces and related infrastructure developed as asynchronous versions of BinaryReader and BinaryWriter from .NET standard library.

Luk164 commented 3 years ago

@sakno I understand that, but it still threw me off while I was trying to figure out what does what. Do you think it would be possible to create a default implementation that could be used out of the box? I currently use BinaryFormatter (not safe, I know but there are alternatives that achieve the same) and just serialize my object into byte[] and send that. In other words, any [Serializable] object can be used.

So instead of extending objects and implementing interfaces a user could just use the default implementation and provide his Serializable object.

BTW the project I am working on is just an assignment from uni. They wanted us to get creative with our implementation so I chose your cluster library as a base.

sakno commented 3 years ago

@Luk164 , it's unlikely from my side to add something called "default implementation" because it gives illusion about simplicity and performance. The price for high-level abstraction is always the performance. This fact is incompatible with the intention to develop performant persistent engine for the cluster.

Anyway, in .NEXT 3.x I have a plan to add the native support of System.Text.Json serialization and another level of abstraction called log entry interpreter based on Command pattern.

Luk164 commented 3 years ago

@sakno That is the point however. With the default implementation you would make it easier for people like me to get started with the library, while still having the more performant approach available for everyone who needs it.