Discussion: advantage to using sync over async for I/O?

[roji]

Continuation of discussion from dotnet/corefx#4868

Without AuthenticateAsClient(), Open() must resort to calling an async method internally, which means that it's no longer purely synchronous... The code above is as synchronous as the old implementation. When you call synchronous I/O on something like NetworkStream, internally, the thread will be waiting on an event for the operation to finish. Even more broadly: in reality there is no such thing as synchronous I/O in Windows.

Thanks for this detail. I was aware that at least in some cases sync methods actually perform async operations under the hood.

However, coming from the Unix universe, it doesn't seem to hold that there's "no such thing as synchronous I/O" at least at the kernel level: I would expect .NET I/O operations to map to simple sync I/O system calls such as write and read. Also, the implementation details underlying async operations across platforms may make sync I/O more attractive at least in some cases (i.e. is async I/O implemented efficiently on Linux with mono, or with coreclr for that matter?). I admit I'm just asking questions, I don't have enough solid knowledge here.

Regardless of these questions, and I said back in dotnet/corefx#4868, there's something odd about removing such a specific sync API (SslStream.AuthenticateAsClient) for such a general "sync I/O is bad" argument, without touching any other sync API. In other words, I'd expect either all sync APIs to stay for backwards compatibility or for a more sweeping removal of sync I/O APIs.

[JeffCyr]

@JeffCyr, are you an employee of Microsoft in some way, stating that "Microsoft/.Net Foundation won't change their mind about this decision now"?

I am not, I might have overstated this last affirmation, this is speculation on my part.

[aL3891]

Is the question here really about sync vs async though? isn't it more about the weight of the Tpl/threadpool model? I do think Gc pressure and thread/threadpool starvation are very valid concerns, but I would prefer a more flexible solution that would allow me to write a single thread, synchronous version (using manualresetevent or whatever) if I wanted too, but also to do something else like my own thread pool with dedicated threads.

Then again if you really really want to maximize performance, maybe dropping to the OS specific apis are the way to go anyway, but it would be really neat to have a lower overhead option for async calls in general as well. Maybe just being able to pass in your own task scheduler to all *Async() calls would be a nice middle ground (I know you can do this in a lot of places already)

The main reason as I see it for the synchronous versions is convenience, but personally I find the .Result to be simple enough :) I do think you're right in voicing your desire for the other sync methods to be added though, the corefx does listen and if enough people want it, i'm sure they'll add it.

[damageboy]

Hi All, I just found this thread and my blood pressure is already starting to rise

I see that in dotnet/corefx#4868, @roji was told by @CIPop that:

Blocking threads on I/O is always a bad programming pattern. To promote good patterns we are trying to remove all sync I/O public APIs.

Later in that thread he was asked to open a separate issue, which he did here, where from what I can tell, I can mostly see different opinions about .NET TP/GC/async/sync being exchanged.

As far as I'm concerned, if @CIPop is not messing with us, he's basically saying what amounts to: "Oh, by the way, we, here at MS and the .NET Foundation, took some hallucinatory drugs and decided that the hell with 40+ years of synchronous I/O APIs, we're dropping it"

Is this real? Can I see a real discussion with the community, initiated by MS/someone in the CLR/CoreFX teams that actually says: "Look, we're thinking about dropping sync I/O APIs, what are your thoughts?" Or is this again the resurgence of the "We know better than all of you approach" that we are supposed to be supposedly over with?!?

Can someone from MS REALLY get on this and state you intentions CLEARLY about sync I/O. Not as a half-assed comment on a missing method in SslSteam, but seriously approach to community and discuss (which I've seen very little of) with IT the pro and cons of what you are thinking of doing, AFTER you've actually declared the scope of the intended change...

In case it wasn't clear that I would like some "official" response: What are you planning to do? Is this a half-assed attempt to remove sync I/O APIs in corefx parts where the sun doesn't shine like SslStream, or have you completely lost it and are actually intending to drop Stream.Write()/Read() because you drank some of nodejs's kool-aid?

What I see here is an active .NET developer which was told to open a discussion about a global, earth shattering change in .NET to remove sync APIs and is getting zero attention as to the claims/questions he is raising from the same people who asked him to open said discussion and others in Microsoft

[JeffCyr]

@damageboy The Stream abstraction is not IO bound, e.g. MemoryStream, so I don't think Stream.Read/Write will go away.

[damageboy]

<cynical> @JeffCyr ah, wonderful, so by induction I can do SslStream over MemoryStream, so SslStream is not I/O bound as well, thanks for clearing that up, so I assume that MS is already in the process of adding SslStream sync support, thanks @CIPop !, Case closed? </cynical>

No offense @JeffCyr, but I would like to hear some response from someone in MS that can actually speak about their entire direction with the duality of sync/async

There's only so much that we, the community, can discuss amongst ourselves with no back-and-forth with MS while some MS devs make wild claims about dropping sync-I/O because "it's bad, don't you know? Someone wrote a blog post about it".

[damageboy]

@JeffCyr btw, you did, in passing, make me think of a coming trend in I/O HW systems, where I/O is actually becoming so fast, that the O/S is increasingly becoming a bottleneck when it comes to I/O performance.

More and more, we are seeing new low-latency I/O components, be it network cards that support user-space (e.g. non-blocking) networking, memory-mapped non-volatile storage (here's a recent BUILD talk about RAM-like storage systems: https://channel9.msdn.com/Events/Build/2016/P466)

It is entirely possible that as the H/W landscape continues to rapidly change, for certain types of I/O async might become too much of an overhead when compared to the fast-path sync option of future H/W...

That is why any such claim as "async is better, no matter what the underlying I/O is" is laughable at best.

I would like to have a serious discussion between whomever is capable of representing the current thought-process of MS is and us, the community...

[GSPP]

Blocking threads on I/O is always a bad programming pattern. To promote good patterns we are trying to remove all sync I/O public APIs.

Yes, this would be a bad policy. It is important to understand how much trouble async APIs are causing. Stack Overflow is full of C# questions where someone had an async-caused problem. The Roslyn commit logs regularly have bugs caused by async programming patterns.

Async is very costly in terms of developer time! It is a complicated trade-off and across the board rules such as "always use X" are not possible.

The .NET ecosystem is still worshiping the async fad. It was much worse 2 years ago. Finally, opinions are getting more nuanced and I hope that the sync/async trade off will be understood more widely in the future.

[roji]

@damageboy and @GSPP thank you for your voices in this, I agree 100%.

[aL3891]

I understand that you're frustrated, but you should know that in my experience the teams generally do not engage in discussions where people call them names (I lurk quite a bit on the dnf repos).

If you want them to engage in a discussion you have to give them more than you've just got nodejs envy and we've done it like this for 40 years. Remember that the whole idea with .net core is to trim the fat and be modular, they want to be restrictive with what they add because once its in there, they can't take it out. You have to provide really compelling arguments to why this should be in the framework and not in say, an external nuget package as extension methods. What specifically makes async (with await or .Result) so much more difficult to use compared to a sync call? What other drawbacks does the Tpl async model have? Allocations and starvation are great examples that's been given already.

@CIPop is a microsoftie so while it might be short of a completely official word from Microsoft I think his views here pretty much reflect the others in corefx teams (correct me if I'm wrong though)

I'm not trying to be a jerk or shut you up, I just think the way you're approaching the issue is not the most productive way to get your way. I do agree that the response from the team has been limited, but lets also remember where they are/where in the release cycle when this issue was raised. I can absolutely see something like this begin added post rtm if enough compelling reasons are given and enough people want it

[damageboy]

@aL3891 I do understand your point, but the parent issue / discussion was "opened" / reported back in December 16th, 2015, since then it was addressed ONCE, by a microsoftie, without really stating the global plans for async/sync I/O APIs after clearly stating, "oh yeah, we're basically dropping sync", here it is again, read it, and imagine you're me, reading this out of the blue, 3 hours ago:

Blocking threads on I/O is always a bad programming pattern. To promote good patterns we are trying to remove all sync I/O public APIs.

The only thing said in that response (19 days ago) didn't address the previous comment, quoted a table from a windows-only networking book from 2002, concluded that I/O will always be slower by orders of magnitude when compared to CPU and left it at that.

I have a long list of objections to many of the things stated there, but before I list them, I would still like to get some response from MS/corefx people as to what their plans are, as one of their team members laid out a claim that they are "removing public sync I/O APIs" without providing any firm details.

I DO understand the copious amounts of pressure the team(s) must be under to deliver a working first release, but it's been almost 6 months with one response, from one corefx team member, that HE asked to OP to open in the first place, to a 60 comment issue/thread filled with non-MS people hyperbole-ing about what the holy spirit of MS means by "removing sync I/O APIs" and why...

Is this (me) being a jerk given the lack of interaction / clarity over 6 months? Waiting patiently sure didn't help, what/how would you suggest I do, as the original comment obviously pisses me off?

[aL3891]

Hey, as you say i'm just an outsider, as most of us all are, and i'm just guessing. As I said i think you and everyone else should speak their mind, imo a frank discussion is always better than no discussion, but that's me. I've had the most luck pinging various people on the team or raising new, more specific issues. Right now it covers at least three things:

• Sync apis vs async when it comes to performance. Starvation/heap allocation, predictability of the thread pool vs os thread scheduling
• Ease of use of async methods compared to sync methods, is the additional syntax/considerations worth it? Should corefx be opinionated and push users towards what they consider correct?
• The consistency of sync apis. Some classes have them, others don't, is this intentional?

Each of these can probably be a whole discussion by themselves. That's not to say this thread is a waste though, its good to start out broadly

[roji]

@aL3891 IMHO we should not really be discussing sync vs async performance, or the ease of use of async methods - these are huge and complicated subjects. The point of this thread as far as I'm concerned is whether sync APIs which already exist in .NET Framework should be removed in corefx, and to a lesser extent (but still important), whether new classes in corefx should continue to provide both sync and async APIs.

Of course this question is related to performance and ease of use. But it's enough to acknowledge that there are some scenarios where sync I/O fits better, rather than start microbenchmarking performance and start huge philosophical discussions. This should ideally be as constrained a conversation as possible otherwise it will become unmanageable immediately.

@stephentoub, @terrajobst, @shanselman any chance you guys can weigh in on this?

[richlander]

Hi! We're in the process of revisiting some API decisions with CoreFX (more coming on that soon). There are two distinct goals:

• Make it easier for developers to adopt .NET Core.
• Enable developers writing the code they want (e.g. generic and non-generic collections, sync and async APIs). No enforced policy from 98052, just helpful guidance.

[roji]

Thanks for the info @richlander, looking forwad to hearing more details.

Somewhat ironically I'd drop non-generic collections as I can't see any advantage they offer (apart from backwards compatibility)...

[richlander]

That would be my point 1 above.

Make it easier for developers to adopt .NET Core.

The presence of non-generic collections, for example, doesn't remove any of the value of .NET Core. For customers that want to re-use code that relies on ArrayList, they can. They are free to move that code to use List<T> when they feel that's a benefit to them.

[ayende]

I agree with @damageboy Async I/O is all well and good, but miss a single ConfigureAwait(false) and suddenly you have a deadlock in your code when it is run in ASP.Net, or WPF app.

Another major issue with async I/O is that they are not debuggable. Let us assume that I've a sync system, and a request is taking a lot of time. If I'm using sync I/O, I can take a proc dump, look at the threads, and say "oh, I see that it is stuck in ResolveDnsAddress or some such", and I know exactly where to look.

If I'm using async I/O and tasks, I'm SOL, because there is just no way that I can track this properly. I know that VS has some stuff that suppose to help with this, but it doesn't help for production, and it doesn't come near what we need to actually support complex systems that may exhibit issues in production.

Another issue is that if you are using slow I/O (which is a good reason to want to use async), your task objects are very likely to end up in Gen1 or Gen2. That lead to more expensive GC pauses, which is something that we really care about.

I really don't like this decision, and what is more, I don't like the fact that it seemed to have been decided in darkness and just pressed onward.

[ayende]

Oh, and what is even more fun, you can't do an await inside a constructor, which means that in some cases, you really don't have a choice but to block.

But there is no really good way to actually do blocking that is out of the box, works across the board and doesn't incur its own set of challenges.

See also: http://journal.stuffwithstuff.com/2015/02/01/what-color-is-your-function/

[JeffCyr]

@ayende

Another major issue with async I/O is that they are not debuggable. Let us assume that I've a sync system, and a request is taking a lot of time. If I'm using sync I/O, I can take a proc dump, look at the threads, and say "oh, I see that it is stuck in ResolveDnsAddress or some such", and I know exactly where to look.

If I'm using async I/O and tasks, I'm SOL, because there is just no way that I can track this properly. I know that VS has some stuff that suppose to help with this, but it doesn't help for production, and it doesn't come near what we need to actually support complex systems that may exhibit issues in production.

It's true that async I/O is harder to debug right now. With sync code you can easily see the threads and their callstack to see what is running. With async code the data is still there, but its scattered across the heap in Task and IAsyncStateMachine objects. I've used ClrMD a couple of times to see which async methods were running in a memory dump by enumerating their associated IAsyncStateMachine. It would also be possible to reconstruct the entire async methods call tree, if Visual Studio did this to display something similar to "Parallel Stacks" it would be awesome.

Another issue is that if you are using slow I/O (which is a good reason to want to use async), your task objects are very likely to end up in Gen1 or Gen2. That lead to more expensive GC pauses, which is something that we really care about.

This is also true, but don't you think that if you used slow I/O, a lot of other objects will end up in Gen2 (like a pending request and all associated objects) and that extra task going to Gen2 won't make a measurable impact.

[roji]

@JeffCyr,

Another issue is that if you are using slow I/O (which is a good reason to want to use async), your task objects are very likely to end up in Gen1 or Gen2. That lead to more expensive GC pauses, which is something that we really care about.

This is also true, but don't you think that if you used slow I/O, a lot of other objects will end up in Gen2 (like a pending request and all associated objects) and that extra task going to Gen2 won't make a measurable impact.

That's really going to depend on your app, but in any case consider the following. If you're just thinking about that one async call, then maybe one Task on the heap seems reasonable. But any serious program is going to have a whole bunch of methods calling each other. When you're doing async, each one of those stack layers is costing a Task on the heap.

Another case that is somewhat more exotic is operations that may perform I/O, but in many cases won't - imagine a case where caching is implemented to save I/O operations. In these cases it's sometimes recommended to cache Task instances rather the results, to save that extra allocation. But that may not always be possible, and again, if you consider that the caching async method is deep down the stack below other async methods which don't cache - you're allocating tons of garbage for a non-blocking operation. Sometimes the effect can be mitigated, sometimes not, depending on what you're doing.

Remember that nobody here is saying "async is bad", just that it has its disadvantages and costs and that sync I/O should remain an option.

[GSPP]

Task-based async is really nasty and infectious. The two things that Go improved with over C are Garbage Collection and Green Threads for IO. It would be so nice if the CLR was 99.99% synchronous but on green threads.

It would be very rare to need to use async IO. You would only need async IO if even green threads were to expensive. That should be in the range of >100000 connected sockets or more. Almost no application in the whole world would need this.

Instead, we are seeing a nasty infection of "async everything" right now. Without Green Threads it's a hard tradeoff what to do. With Green Threads the decision almost always is easy: Use sync IO and block in order to not infect the caller.

[clrjunkie]

@ayende

Async I/O is all well and good, but miss a single ConfigureAwait(false) and suddenly you have a deadlock in your code when it is run in ASP.Net, or WPF app.

Suggest project-specific default setting for this? (e.g library project)

Another major issue with async I/O is that they are not debuggable

Open a separate issue for this? Advocate for including APM counterpart API's ?

Oh, and what is even more fun, you can't do an await inside a constructor,

Advocate for a language enhancement?

What does async/await have to do with Sync Programming model vs Async Programming model..??

[ayende]

@clrjunkie Because you now asked me to open three difference issues, all of which start with -100 points. While having those is nice, I want to know how async everything got passed that same hurdle without even seeming to think about it.

[clrjunkie]

@ayende

Because you now asked me to open three difference issues, all of which start with -100 points.

Didn't get it, is there a point system here?

I want to know how async everything got passed that same hurdle without even seeming to think about it.

I'll give you my take once you clarify, as I'm unclear on the "same hurdle" part.

[ayende]

@clrjunkie If all features start at -100, what made dropping sync pass that into "let's go for that". Once you have that, just to be back in the same place, you need many other quite complex things.

[aL3891]

The points things doesn't really apply for removing stuff though, I think the way to think about it is that sync apis was a feature that also started at -100, and didn't make the cut the first time around. But again, the team are reevaluating that stuff as we speak

[ayende]

@aL3891 if you are removing stuff, you have better had a good something to replace it. When you give async only I/O, you are losing a LOT of stuff. That need to be replaced.

Now, it made sort of sense when it was Silverlight and the browser, but when talking about "port you existing systems", that is a big bad thing to do.

[aL3891]

Well different people hold different opinions regarding what's good and what has really been lost :) i'm just describing what I think their reasoning was. if enough people want it and make a good case for it, they'll add it, as it seems they are in the process of doing right now

[clrjunkie]

@ayende C’mon.. for all intents and purposes .NET Core is a rewrite of .NET, sure they got a head start by having the full .NET for reference and code reuse but do you really believe that the same people who wrote the code 10 – 15 years ago are same people doing the rewrite? Tell me, have you ever tried to study full .NET FW source code? Most if not all the I/O class have two different code paths with hoops and loops and context classes that will twist your brain, not to mention the dependency spaghetti, yes, same as you would see in any other large old codebase.

I can totally understand why they decided to go with async/await, only for the sake of simplifying the port for them.

Implementing one code path instead of two is simpler and simplicity is an engineering criteria.

Now the problem with this issue is the title because all what I’m reading here is people complaining about async/await short-comings thinking somehow that Sync is the solution, but that’s not necessarily true (heck I complained about it to a dead end dotnet/runtime#15864). Sync is a different Programming Model and I didn't read here even one scenario to justify it.

"allocation", "ThreadPool pressure" are not Sync Api scenarios!!

I’m not arguing Sync doesn’t have value but if it was up to me I wouldn't invest a dime in Sync unless I see a concrete use-case that shows a noticeable difference over async!

So till then, if your immediate problem is due to async/await – shout FIX ASYNC AWAIT!

[roji]

for all intents and purposes .NET Core is a rewrite of .NET

Maybe the implementation is a rewrite, but with regards to APIs and contracts it really isn't - backwards compatibility with .NET Framework is a major (if not absolute) goal of .NET Core.

Implementing one code path instead of two is simpler and simplicity is an engineering criteria.

@clrjunkie the cost of maintaining both a sync and an async path in any library is indeed high - you're right. I faced this exact same problem in Npgsql and ended up writing a code generator called AsyncRewriter for creating async methods from sync counterparts. There's nothing pretty about this but it does get the job done - Npgsql fully supports both sync and async I/O without me maintaining 2 full code paths.

But at the end of the day sync I/O simply isn't some small self-contained piece of functionality that can be dropped for simplifying the port for the MS team.

Note that nobody at MS ever raised this argument - "we're dropping sync because it's too much work for us"). Nobody at MS even said "we're dropping sync", almost all sync I/O APIs are still there. It's just a sync method here and there that's missing, with a vague and uncommitted "sync is bad" argument.

Now the problem with this issue is the title because all what I’m reading here is people complaining about async/await short-comings thinking somehow that Sync is the solution, but that’s not necessarily true (heck I complained about it to a dead end dotnet/runtime#15864). Sync is a different Programming Model and I didn't read here even one scenario to justify it.

You're looking at it the other way around. Sync isn't a "solution" to async's problems! There are simply some scenarios where async provides no advantage over sync (e.g. no scalability requirements) and only makes things worse.

Also, some of async's "problems" are inherent in the programming model - they're not something that can be fixed (your suggestions notwithstanding). And even those things that can be improved or fixed aren't there yet now.

If after this thread you still refuse to admit the async has its costs and that sync can sometimes make sense - heap allocation and GC pressure, programming model complexity, TP pressure - then you're just stuck in an ideological "async all the things" loophole that doesn't care about actual facts and programmer concerns.

[clrjunkie]

@roji

backwards compatibility with .NET Framework is a major (if not absolute) goal of .NET Core.

Who said so? Where did you read this??

there are simply some scenarios where async provides no advantage over sync (e.g. no scalability requirements) and only makes things worse.

No problem, since you argue so strongly about what you consider facts and programmer concerns I challenge you to paste here 2 short console applications (async/sync) which do something useful for you and clearly show how sync outperforms async. That way we can discuss an actual programming use case instead of buzz words.and ideology. Ready?

[roji]

backwards compatibility with .NET Framework is a major (if not absolute) goal of .NET Core.

Who said so? Where did you read this??

Read this blog post by Immo Landwerth, and especially the section "Relationship of .NET Core and .NET Framework". .NET Core isn't a new "let's break-everything" effort.

Note also how "technologies discontinued for .NET Core" are explicitly discussed and analyzed (app domains, remoting...). Note how "sync I/O" is not listed in that list.

there are simply some scenarios where async provides no advantage over sync (e.g. no scalability requirements) and only makes things worse.

No problem, since you argue so strongly about what you consider facts and programmer concerns I challenge you to paste here 2 short console applications (async/sync) which do something useful for you and clearly show how sync outperforms async. That way we can discuss an actual programming use case instead of buzz words.and ideology. Ready?

I'm sorry, I'm done wasting my time with you on this - me (and others) have already given examples in this thread many times over. Avoiding the increased heap allocation associated with async is an actual concern for at least some programmers, it's directly linked to your application's performance. Until you answer here why programmers shouldn't be concerned about heap allocations and GC pressure (and the other issues listed above) there's no point in continuing this conversation with you.

[roji]

@clrjunkie, regarding compatibility between .NET Framework and .NET Core take a look at the corefx breaking changes guide, you'll understand to what extent Microsoft is trying to make .NET Core compatible.

[ayende]

For further discussion, see this PR: https://github.com/dotnet/corefx/pull/8642

Hibernating Rhinos Ltd

Oren Eini* l CEO l *Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

On Tue, May 17, 2016 at 3:25 PM, Shay Rojansky notifications@github.com wrote:

@clrjunkie https://github.com/clrjunkie, regarding compatibility between .NET Framework and .NET Core take a look at the corefx breaking changes guide https://github.com/dotnet/corefx/blob/master/Documentation/coding-guidelines/breaking-change-rules.md, you'll understand to what extent Microsoft is trying to make .NET Core compatible.

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/dotnet/corefx/issues/5044#issuecomment-219701923

[clrjunkie]

@ayende For whatever reasons, this is at most an example of a problematic API design as the file handle is not polymorphic and there is a clear change in the API functional semantics depending how the object was constructed. Furthermore my understanding from the issue is that in this implementation Sync is simulated by waiting for Async results to complete and there is no O/S sync call involved. So the issue itself is not quite relevant to this discussion.

Saying that, I would argue that synchronous File I/O can be more efficient when the access pattern involves many small read/writes. I actually thought about posting here a sample program to demonstrate this, but I haven’t found a real world use-case that’s important or general enough and I didn't want to use some artificial write "counter" example. I would bet that a work queue of fixed dedicated threads issuing many small writes per work item synchronously would process items faster than submitting those writes asynchronously. (assuming of course the number of work queue threads correspond to the number of CPU’s, again one work item yields many small writes). You are welcome to find and test a business use-case that fits this model and submit it to justify the need.

[clrjunkie]

@ayende

Because you now asked me to open three difference issues, all of which start with -100 points, While having those is nice,

Coming to think about it, even so, if you find these truly nice then submit them, maybe they will get in someday, at least they will be documented, so do that. (No cross-reference required :)

[benaadams]

Saying that, I would argue that synchronous File I/O can be more efficient when the access pattern involves many small read/writes.

Nah; as you can double buffer with async and you can't with sync. So you can be preparing the next batch of data while the current one is being written out breaking the read+write dependency; and giving the task time to complete before you await it, so its usually a completed task.

e.g. the throughput on

private async Task CopyToAsync(Stream source, Stream destination, int bufferSize, CancellationToken cancellationToken)
{
    var buffer0 = new byte[bufferSize];
    var buffer1 = new byte[bufferSize];

    var readTask = source.ReadAsync(buffer0, 0, buffer0.Length, cancellationToken);
    var writeTask = Task.CompletedTask;

    var isOdd = false;
    int bytesRead;
    while ((bytesRead = await readTask.ConfigureAwait(false)) != 0)
    {
        readTask = source.ReadAsync(isOdd ? buffer0 : buffer1, 0, bufferSize, cancellationToken);
        isOdd = !isOdd;
        await writeTask.ConfigureAwait(false);
        writeTask = destination.WriteAsync(isOdd ? buffer0 : buffer1, 0, bytesRead, cancellationToken);
    }
    await writeTask.ConfigureAwait(false);
}

will be much higher than

public void CopyTo(Stream source, Stream destination, int bufferSize)
{
    var buffer = new byte[bufferSize];
    int read;
    while ((read = source.Read(buffer, 0, buffer.Length)) != 0)
    {
        destination.Write(buffer, 0, read);
    }
}

Alas the framework doesn't do CopyToAsync this way as its more difficult to attribute failure states, e.g. what failed first.

[GSPP]

@benaadams I tried that with buffered IO. Throughput was not better because the kernel does read-aheand and write-behind in the kernel.

Also, if this was beneficial you could get the same benefit with sync IO and threads at a slight expense of CPU time.

[ayende]

@clrjunkie

Features implemented years from now have very little use at this point.

Hibernating Rhinos Ltd

Oren Eini* l CEO l *Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

On Wed, May 18, 2016 at 4:26 PM, clrjunkie notifications@github.com wrote:

@ayende https://github.com/ayende

Because you now asked me to open three difference issues, all of which start with -100 points, While having those is nice,

Coming to think about it, even so, if you find these truly nice then submit them, maybe they will get in someday, at least they will be documented, so do that. (No cross-reference required :)

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/dotnet/corefx/issues/5044#issuecomment-220024892

[clrjunkie]

@benaadams Nah; Your biased on a particular processing model.

Each Thread:

var count= fileBatch.Count;

for(var i = 0; i < count; ++i) { write(fileBatch[i]); }

[clrjunkie]

@ayende

Features implemented years from now have very little use at this point.

First, you don't know what priority these feature will get. Second better late then never :)

[clrjunkie]

@benaadams You are assuming that items are always prepared and persisted concurrently. There could be is a time gap between the two for some use-case. I'm not ruling out any use-case.

[clrjunkie]

Buffering log records in memory into buckets - delimited, not concatenated - and submitting them to be flushed concurrently to a set of new files after X ? Ex: A high performance logging solution that trades-off 100% durability for speed?

[roji]

@clrjunkie, regarding the question of API compatibility between .NET Core and .NET Framework, please see this blog post from yesterday. I think that pretty much closes the discussion about whether sync I/O APIs will be added. Of course you can still claim they're useless and bad.

EDIT: Fixed link

[clrjunkie]

@roji Thanks for the heads up. (though the link you posted self references this issue..!@? )

I assume you meant this:

https://blogs.msdn.microsoft.com/dotnet/2016/05/27/making-it-easier-to-port-to-net-core/

Well, my only "claim" is that it appears the same "rules" apply - See ya' in .NET Core V3.

Until then...Count me out on this.

-Unsubscribed-

[roji]

@clrjunkie thanks, fixed the link.

@CIPop and others at Microsoft, I'll leave this open to track bringing back SSLStream's missing sync methods, feel free to close.

[CIPop]

Thanks @roji. The scope for API surface changes that are being considered is broader than SslStream. Regarding the async/await debugging and performance issues, I recommend opening separate issues.

Assigning this discussion to @terrajobst to track the necessary API changes.

[karelz]

@roji can you please summarize the ask? Ideally in a new issue - this one has rather long (finished) discussion hard to reason about, I would prefer to close it.

Note that the APIs will be also considered as part of .NET standard 2.0 effort (see more here and here).

[roji]

@karelz - the request is simply for BCL APIs to include synchronous APIs alongside asynchronous APIs, in order to support the sync programming model. A special emphasis should be made on bringing back sync APIs which exist in .NET Framework but were dropped in .NET Core 1.0 (e.g. SSLStream.AuthenticateAsClient()).

With the shift towards making .NET Core more API-compatible with .NET Framework to help porting, I'm not sure another issue is necessary here for that - although if you think so I'll open one.

[karelz]

Agreed that a new issue is likely not needed at this point. Thanks!