JasperFx / wolverine

Supercharged .NET server side development!
https://wolverinefx.net
MIT License
1.21k stars 131 forks source link

Redis Streams Transport? #507

Open AndyPook opened 1 year ago

AndyPook commented 1 year ago

This seems like an obvious extra implementation. The options in src/Transports are great, but they do require significant effort to plan and support in production. Especially if the only need is for Wolverine.

I do plan to try creating this myself... Question for the community... is anyone else doing this? I would prefer not to duplicate the effort.

Question for Jasper... Is there any guidance on what is required? I have been looking at the various implementations. It is not clear (to me, yet) how the abstractions work together. The use of new in various places gives pause. Stub, Local and TCP places to start. Digging into some of the "full" impl's, there's a lot going on that bring in more parts. There's a lot that seem to represent that transports basics (ie the various "Internal" folders). Would this be the normal approach? Or would something more like Pulsar (facades over an external lib) be appropriate for a Redis impl?

Thanks for any help/guidance

jeremydmiller commented 1 year ago

Hey, I was unusually busy last week, but I just barely looked at this. I don't know that Redis would be all that similar to the existing transport. You might look at any of the "broker" transports like Azure Service Bus or Rabbit MQ to see the pattern. I wouldn't look at TCP or Local at all for this one.

AndyPook commented 1 year ago

Forget I mentioned redis 😃 What would be really useful is just an understanding of the abstractions. For example, when you say "broker", does that just mean "external message store"? (I wouldn't call mssql a broker, but it implements BrokerTransport) There are a lot of interfaces and base types floating around. It's hard (for me) to get a grip on how they relate, when they are useful, when not.

Rabbit and AzureSB implementations are... a lot 🤔 Whereas Pulsar appears to be more "simple" facades of transport, sender, listener, mapper etc It does not seem to use the "broker" bits, but is more like an actual broker than some of the others. Why is the Pulsar example so different than the others?

Trying to build a mental model of what is required, be it redis or anything else. I'm looking to use something more "light weight" than the existing transports (redis just seemed a likely choice). But if rabbit is a good example, then that would probably take more "investment" than I can give (in the scope of the current project).

I do appreciate that these things are hard just trying to get a handle on how many and how steep the necessary learnings will be

kurtisane commented 4 weeks ago

That's sad.

I was looking forward for wolverine with redis streams. It would be the first library supporting that.

We have currently our own implementation via Background Services, but it would be ofc awesome to use something existing.

I would really appreciate a explainer of the abstractions than as well ( like @AndyPook mentioned ) so that maybe the community could create something fitting.