search

bumptech / nitro

Queueing/Networking Library
Other
112 stars 16 forks source link

Consider documenting rationale and comparisons #3

Open istathar opened 11 years ago

istathar commented 11 years ago

@jamwt,

Can you add some text to one of your documents explaining your rationale for developing nitro and/or comparing to some of the other event/channel/message passing frameworks out there?

I'm wildly impressed by what I see, but I immediately tripped over the "why are they developing this, anyway" question.

It's already clear to me that using this would be an alternative to Cloud Haskell, but that raises the question of what architecture you'd use this in and what your cost model is.

AfC

dowski commented 11 years ago

@afcowie Have you seen the comparison to ZeroMQ in the FAQ?

istathar commented 11 years ago

@dowski so I found that, quite by accident, at the bottom of that really rather long document. So, yeay, but you might consider splitting that page into several, with API documention separate from descriptive, introductory, and FAQ text.

AfC

edahlgren commented 11 years ago

@afcowie What led you to this repo? If http://hackage.haskell.org/package/nitro, it occurs to me that the motivations in the FAQs that @dowski references could be made more accessible there too.

istathar commented 11 years ago

@edahlgren yeah, it was the Haskell package.

And just to re-iterate, much as I like the foundations of Cloud Haskell, I think in practice [so far] it's quite limiting because of the insanely tight ABI level coupling. So even if I was working in Haskell only (a big presumption and indeed why limit yourself) I'd still be more likely to use an external queue mechanism.

Which raises another question: do you see nitro as a queue or a transport? By queue I mean more like Amazon's SQS; by transport I mean pub/sub mechanism on top of TCP. I'm guessing the answer is both, but SQS for example has an insanely strong durability guarantee but delivers out of order across AZs, etc. Cloud Haskell (not that it's apples and oranges here) is very focused on ordered delivery.

Anyway, nitro looks terrific. I'm trying to help you talk me in to using it :)

[all toward helping you improve the documentation]

AfC

edahlgren commented 11 years ago

@afcowie You can think of TCP sockets as ordered queues if you want, and if you set a hwm, then as bounded ordered queues. A big point for us: Nitro is transparent about how many packets (we call them Nitro "frames" = routing info + message) are sitting in these TCP/Inproc queues.

There are many other high level selling points in those FAQs. Email one of us directly if you have further questions; we'll try to improve the documentation based on feedback.

magicseth commented 11 years ago

I don't think that's actually true @edahlgren Tcp sockets are ordered queues, but a nitro socket may be composed of multiple tcp sockets in time. For example if the socket is dropped, it may reconnect. Therefore, packet delivery is not guaranteed and packets may disappear from the middle of the "queue".

edahlgren commented 11 years ago

@magicseth Yes, packets may drop out of the "queue" if the socket is dropped, so nitro can't make a guarantee about durability. It's a client's job to resend their packet if they don't receive a response from a server, etc, which is nothing new. My point was only that TCP does the the ordering.

istathar commented 11 years ago

@edahlgren ok, so that makes it clear to me that nitro is a transport not a queueing system. Cool. Now to figure out what to use if I were to need a durable queue and wished to use nitro as primitive.

I think you can close this, except for the original point that the Intro & FAQ material should be on their own page(s).

AfC