Closed gfoidl closed 2 years ago
Again a long read, and again very impressive. I didn't even get to the actual code yet, but all your research is really nice. Love the tiny details like the int return for field reuse. Will start reviewing this on phone, so I might not finish it today. May I ask how much time you spent on all this? Again, impressive feat, I feel honored by all the effort you're putting in.
Well it wasn't much to review
Lot's of text for a little bit of code change, haha 😄
May I ask how much time you spent on all this?
I didn't work in one rush on it, but started on Saturday after lunch, and the findings are quite interesing. The thing with the "int-return" re-use is also a TIL-moment for me...so I didn't monitor the time taken, but it's a couple of hours (full of passion).
First off I whish you and your child all the best -- get well soon! (So sorry that this text got long...)
This PR addresses the points brought up in https://github.com/mycroes/Sally7/pull/23#discussion_r730095872.
Abstract
ValueTask vs. custom SocketAwaitable
To investigate which option seems better as first step there's an reduced benchmark, that sends a buffer via loopback to a server, which in turn reads the sent data.
Benchmark with simple read and write to loopback
It's really not so easy to get reliable results from the benchmarks, due all the latencies involved. So I've run the benchmarks and profiling sessions multiple times, and tried to pick the results from a "representative" run (i.e. not the best, not the worst ones).
With some debugging (and hacky console output) it can be seen that send completes synchronous most of the time, while receive completes async most of the time. So I hope this benchmark / comparison is fair, and doesn't bias the results (too much).
When run
bench.WithSocketAwaitable()
in a loop of 100.000 iterations with BufferSize set to 960, a profiler showsSo even we do 100.000 iterations, there are less than that allocations for the AsyncStateMachineBox, meaning that some operations (send, receive) complete synchronous. When the profiling session is run multiple times, these number change quite frequently. That was already alluded above.
For the
SocketAwaitable
I tried a version withICriticalNotifyCompletion
too, that avoid the ceremony with the ExecutionContext, but results didn't change much (whether positive nor negative).The
AsyncStateMachineBox
is needed to let the locals of the async-method survive, i.e. they need to be allocated on the heap.The same with
bench.WithValueTask()
yieldsSome rans show the saw-tooth pattern like above too, but for several runs it looks more like the allocation graph shown in the image. I have no reason why this behavior is the way it seems to be.
The
AsyncStateMachineBox
is needed for the same reason, to let the locals survive by allocating them on the heap.In the VT-based version the allocations for the
Action
never showed up. I'd count this as positive point. So if we assume that send or receive complete the same amount asynchronous as with theSocketAwaitable
-version, then we can save the allocations for the action-delegate.But why isn't this delegate needed for the VT-version? It's because .NET has for VTs a special
AwaitableSocketAsyncEventArgs
which implement severalIValueTaskSource
s for the VT, and (most importantly) these instances are cached. So there's no need to allocate the action delegate for the continuation over and over again.In the following figure focus on the marked are of the calltree.
Both start with the IO completion from the OS by executing
PerformIOCompletionCallback
. Then the difference begins. For the VT (right hand side) there's theAwaitableSocketAsyncEventArgs
plumbing things together for the ValueTask via theIValueTaskSource
-methods, and only the allocation for the statemachine-box is needed. This is because the ValueTask is backed by the IValueTaskSource, and not a Task for the async completion. I'll skip over the exact details on how this works, the code is on GitHub :wink:. On the other (left) side there's the calltree for the SocketAwaitable, allocating the statemachine-box, and also the "MoveNextAction" forAsyncTaskMethodBuilder<T>.AwaitOnCompleted
.So based on that investigation I'd go with the ValueTask where available.
Benchmark code
```c# using System; using System.Diagnostics; using System.Net; using System.Net.Sockets; using System.Runtime.InteropServices; using System.Threading.Tasks; using BenchmarkDotNet.Attributes; #if !DEBUG using BenchmarkDotNet.Running; #endif #nullable disable Bench bench = new(); bench.Setup(); await bench.WithSocketAwaitable(); await bench.WithValueTask(); #if !DEBUG BenchmarkRunner.RunBenchmark against a soft-plc
Let's see how it looks when reading with Sally7 from a PLC. As PLC softplc is run in a linux-VM with some test-data applied to it. To have a benchmarks that runs both variants (default and with ValueTasks) in "one run", I duplicated the SocketTpktReader to have the default version, and one that uses ValueTasks. Other classes are duplicated as needed. See commit 126259d8f9d57a52515837e8188f6fa277dc3d31. The benchmark just reads three dataitems out of the PLC. So results may and will vary depending how much is read, and especially what the network will do (latencies).
Side note: I believe there's some room for improvements in the handling of the Requests. Will address this in a separate PR, so we can discuss this over there (I like PRs that focus on one point (although I'm aware that I didn't follow my own rule here...)).
Again the note of the flakiness of the results. I tried hard to get some stable results, this is what looks the most realistic:
TBH this matches with my expectations, though I'm a bit surprised that it matches :wink:. For perf / time I guess it's less allocations, and the pre-pinned memory. For deeper analysis of that, I could look with PerfView into this (but not yet).
Let's look with a profiler into it, by doing 20.000 reads.
SocketAwaitable:
ValueTask:
For the
AsyncStateMachineBox
it's the same* count of allocations. But the allocation count for Action-delegate halved from 40.000 to 20.000.* the tiny difference results from sync vs. async completion of a socket-operations, but they are pretty close, and one time it's on that side, one time on the other side.
20.000 action allocations can be eliminated from both version. When we complete the request, then https://github.com/mycroes/Sally7/blob/e250d37adb2926a6c4e92084d9979edd93b10edd/Sally7/RequestExecutor/ConcurrentRequestExecutor.cs#L235-L237 the
length
is set, butIsCompleted
is left tofalse
. Hence the async machinery has to got the routeGetAwaiter() -> IsCompleted -> OnCompleted() -> continuation() -> GetResult()
, and therefore an Action-delegate needs to be allocated forcontinuation
. By updatingIsCompleted
the route can be shortened toGetAwaiter() -> IsCompleted -> GetResult()
, and not delegate forcontinuation
needs to be allocated.Note: this is a best effort solution, where the request to be awaited is the same as the one handled by the receive (I believe this is the common case). Otherwise there will be the async route.
SocketAwaitAble (after reduction of action allocations):
ValueTask (after elimination of action allocations):
Side note (and a bit a note / remainder for myself):
Before ValueTasks came into .NET I used "await anything" quite extensively, but then somehow I forgot about this nice technique. The code in Sally7, and especially this investigation remindeds me to use this in the future a bit more. There's no need to artificially complicate things with ValueTasks, especially when the "Awaitable" can be pooled / cached easily.
Out of interest I tried different versions of the Request-type.
Default
is the one used in current main-branch.DefaultFixed
is the same as Default, but with theIsCompleted
-fix -- so the one used in this PR.DefaultFixed2
is more or less for reference, and completely synchronous -- this won't work with concurrent requests though, hence "for reference".ValueTaskBased
is based onIValueTaskSource
instead of the custom awaiter.DefaultFixed
with synchronous completion is pretty fast.ValueTaskBased
shows more overhead than the simple custom awaiter, so if possible* use custom awaiters over ValueTasks (of course measure results before taking a conclusion).* one also needs to take ExecutionContext into account, if needed, so the way via IValueTaskSource and the ManualResetValueTaskSourceCore is preferable, as otherwise thing get complicated pretty soon.
Code
```c# using System; using System.Threading.Tasks; using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Running; Bench bench = new(); MemoryThings look different if
await req
doesn't complete synchronous.The ValueTaskBased-version (baked by IValueTaskSource) doesn't need the allocations for the continuation (I believe C# compiler has special handling for built-in awaiter-types). Here the time needed isn't relevant, as the socket-operations will be dominant. It's solely about allocations here.
Benchmark code
```c# using System; using System.Threading; using System.Threading.Tasks; using BenchmarkDotNet.Attributes; using BenchmarkDotNet.Running; Bench bench = new(); MemoryI implemented this after writing the rest of this description with commit c8b312321120c031415d4427b40a607dd1676342. So this isn't written in chronological order. Looking at the IL and allocation profile, this is a regression, as the AsyncStateMachineBox increases by a field for the
ConfiguredValueTaskAwaitable<Memory<byte>>
. 4.479.552 / 19.998 = 224 bytes per instance So after (next section) a reduction of the size, we would increase it again. Thus this change is reverted inThen I got the idea to (try) reduce the size of the AsyncStateMachineBox, by eliminating the field for the
req
.Before:
4.479.104 / 19.996 = 224 bytes per instance
After (commit c718f72cb76ae033eeaf7be3713dd541191de77b):
4.318.488 / 19.993 = 216 bytes per instance
The difference of 8 bytes is exactly the size of the reference we save (on 64bit process). Not a huge saving, but still a little bit.
The size of the AsyncStateMachineBox can be further reduced by letting the C# compiler re-use some fields for the TaskAwaiters. In the ILSpy-images above
<>u__1
and<>u__2
get collapsed into only one field.After (commit da12e690b4ad34d4d9dd68c15107f407d3a897bf):
3.999.200 / 19.996 = 200 bytes per instance
So in total we saved 24 bytes per send / receive operation.
BenchmarkDotNet (BDN) has a heuristic to run as much iterations until the results become stable. You see that effect here. But with
MaxIterations
(or that like) this can be limited -- it's sometimes quite handy, otherwise it may take really long...Yeah, unfortunately it's quite hard to come up with reliable and reproducable benchmarks for this. I don't have any good idea either. Maybe create a soft-plc as a docker image, which can be used for benchmarking against? (also maybe create a separate issue tracking this, so that other people can chime in)
The code here in Sally7 is very well arranged, so I was able to learn about the protocol by skimming through the code :+1:.
PS: I didn't proof-read the text, so please apologize any / all mistakes.