Question regarding "Flush" - Githubissues

RuedigerMoeller / fast-cast

hp low latency reliable multicast messaging

GNU Lesser General Public License v3.0

122 stars 17 forks source link

Question regarding "Flush" #8

Open dmart28 opened 8 years ago

dmart28 commented 8 years ago

Hello,

Having question: say I do some number of "publisher.offer(..)" calls. After that, I just call "publisher.flush()". How much guarantee is that everything offered before "flush" call was really sent down to socket?

Just it seems it's not the case that after flush everything gets written down to channel now from my tests. Thanks in advance.

RuedigerMoeller commented 8 years ago

fast-cast tries to optimize high frequency traffic by packing several messages into a single packet (without adding another thread/queueing). The intended behaviour is:

If flush is set to false, the message is appended to the currently "active" packet, and the packet is sent only in case it is full. In case no new message comes in for a configurable time (default is 1 ms afaik), an auto flush is done.
If flush is set to true, the currently "active" packet is sent immediately after appending the new message.

If you can provide a reproducing testcase (e.g. with messages sending a timestamp) I'll fix that.

RuedigerMoeller commented 8 years ago

Forgot an edge case

in case the message rate exceeds configured pps withing ppsWindow (packet rate limit exceeded), the message is not flushed regardless of flush settings. Assumption is that an application prefers batching over blocking/slow down in message rate peak situations, however this could lead to millisecond level latency spikes in case of a sudden messages rate peak with a sudden end.

dmart28 commented 8 years ago

Yes, so it explains something. It seems I really faced edging condition. Appreciate for your explanations - though you can start doing some Wiki here on top of that questions :)

RuedigerMoeller commented 8 years ago

it should only delay for a millisecond or so, do you observe larger delays ?

dmart28 commented 8 years ago

Haven't observed yet. How ppsWindow actually calculated now? I noticed it is marked as Deprecated.

RuedigerMoeller commented 8 years ago

oops, I forgot about that :). Has been a while i was into fast-cast ..

Ahh . got it again .. I improved that:

originally one could determine the window size (ms) where packet rate is controlled. This could lead to errors if you run high volume traffic as it was possible to overload network buffers without even running into rate-limiting check.
see BatchController.java: it probes minimum nanoSecond timer resolution and chooses this as time window to apply rate-limiting, so ppsWindow is now automatic, you only have to care about pps setting.

RuedigerMoeller commented 8 years ago

btw: are you using fast-cast inside your "reveno" framework ? Looks interesting .. I'll provide first class support on fast-cast, as I don't know of a production-grade system using it yet (my major client seems to stick on fastcast 1 forever, no problem => no upgrade ;) ).

Be aware that multicast does not work on public cloud platforms, though.

dmart28 commented 8 years ago

Yes, I am using it here as one of the options for failover. I liked mostly fast-cast for things like zero-copy, latency in mind. I haven't found another reliable zc, low-lat (with really all that things in box) multicast framework, to my big surprise. Also I am aware of multicast limitations on some environments, etc., so trying to provide multiple options in Reveno, which started recently more as experiment, now had only few releases, so can't say it's very production-grade for now :) Thanks for the support thing!

RuedigerMoeller commented 8 years ago

There is also Aeron, which is probably more advanced than fast-cast, in turn fast-cast is simpler. I compared fast-cast with (an early version of) aeron, and found aeron is slightly faster in the average case (~3-7%), while fast-cast had fewer outliers in the 99.x percentiles. Issue is, that in practice most performance is lost in processing/decoding, so benchmarking pure byte array transport is only a small part of the story. I also used JGroups, which has the advantage of being able to transparently switch to TCP, however the reliable mutlicast implementation is abysmal. I wrote fast-cast because JGroups just was not good enough to support our project. It also does not scale well (>50 nodes clusters), and jgroups tcp stack does not scale well as tcp does not scale :).