search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.56k stars 4.54k forks source link

Networking stack - Technical roadmap #43495

Closed karelz closed 3 years ago

karelz commented 7 years ago

Summary

Our investment in the .NET networking space will focus in the following areas

Foundation: Sockets, SSL, DNS support.

Primary Goal: Provide near-native performance and rock-solid reliability.

Key investments:

Web Stack: HttpClient, ClientWebSocket

Primary goals: Ease of use; feature-rich; extensible; track emerging standards; match or exceed competitor's performance.

Key investments:

Tracking: https://github.com/dotnet/corefx/issues/21452

Emerging Technology

Primary goals: Demonstrate leadership in support for new protocols and capabilities

Key investments:

Maintenance Components: *WebRequest, Mail, HttpListener

Primary goals: Preserve existing customer investments Key investments: None.

Details

(1) Foundational

Key components

Characteristics

Requirements

Current Status

Windows

Linux

Key investments to make

(2) Web stack

Key components

Characteristics

Requirements

Current Status

Windows

Linux

ASP.NET

Key investments to make

(3) Legacy/maintenance components

Key components

Characteristics

Key investments to make

(4) Future/bleeding edge components

Not covered in this version.

Key Feedback

TODO (@karelz)

Credentials & Status

Author: @geoffkizer

Reviewed by CoreFX team: @stephentoub @davidsh @CIPop @Priya91 @wfurt @DavidGoll @karelz Reviewed by partner teams experts: @davidfowl (ASP.NET), @Tratcher (ASP.NET), @mikeharder (ASP.NET), @marek-safar (Mono/Xamarin), @mconnew (WCF), Windows networking experts, @tmds (RedHat), @omajid (RedHat) Reviewed by community networking experts: @benaadams, @NickCraver, @onovotny, @mgravell, @Drawaes

Reviews happened in stages in April. Feedback will be incorporated in updates to this doc. The delay between original review and publishing the roadmap now is purely lack of time to incorporate feedback into the doc & publish it.

Current status: Community review / feedback

Announcements: Twitter and CoreFX repo

mgravell commented 7 years ago

Question: is it relevant / appropriate to ask how "pipelines" fits into this picture? does pipelines fit into this picture? or is that a completely separate picture? and if so, how the two pictures (and timescales) interplay with eachother? hmmm... too many picture metaphors, but... whatever - you get the picture dammit!

galvesribeiro commented 7 years ago

@mgravell was about to ask the same thing... Where does pipelines fit in those cases? In the middle between the foundation and the upper layers?

Aaronontheweb commented 7 years ago

For RIO: isn't that based on Win32 features? Is there an x-plat story for it?

Drawaes commented 7 years ago

I think the idea there would be a pluggable transport layer like kestrel is using, allowing platform optmisations while protocol handling etc staying the same. RIO is a windows only thing.

galvesribeiro commented 7 years ago

@aaronontheweb There is RapidIO driver and NIO/MTPL on linux as well which are somehow similar to RIO in some aspects.

karelz commented 7 years ago

Re: Pipelines

I personally view Pipelines and Span<T> as lower-layer primitives. When they are successful and bring non-trivial wins to the networking stack, we will definitely use them. We keep them in mind as we work on the new stack, but the work is happening mostly in parallel at this moment. They are additive improvements (potentially adding more API surface).

Re: RIO

The idea is to first explore how we could use it / surface it. With that comes obviously also exploration of Linux alternatives (e.g. RapidIO mentioned above). We did not do the investigation yet (neither on Windows, nor on Linux alternatives), so we do not have concrete plan in place.

Aaronontheweb commented 7 years ago

thanks @karelz @galvesribeiro - great to know! Would love to see this make it into DotNetty in the future.

Tornhoof commented 7 years ago

Maybe add the hint that mailkit is the suggested replacement for system.net.mail?

karelz commented 7 years ago

That's a known thing already: https://github.com/terrajobst/platform-compat/blob/master/docs/DE0005.md (from our official under-construction docs)

[UPDATE] Top post updated with the link

Suchiman commented 7 years ago

Sounds great 👍 Does the managed HttpClient also solve these issues or at least not make you tap into them by default? You're using HttpClient wrong and it is destabilizing your software Singleton HttpClient? Beware of this serious behaviour and how to fix it

karelz commented 7 years ago

The managed HttpClient will try to do its best to keep exactly same API surface as the old one (to ease up migration and to eventually make it the default in .NET Core). If the problems above are caused by the API surface, then it won't change.

If we find that by changing the API surface we can get significant perf wins or usability wins, we could eventually entertain the idea to create new API surface (similar to the old one). However, it will be only last resort solution which would have to be paid by significant wins for developers.

ewinnington commented 7 years ago

Where does WCF fit into the dotnet networking stack?

bgever commented 7 years ago

For ASP.NET the single most empowering new feature would be server push with HTTP/2. It significantly enhances performance of SPA apps where critical files can be pushed faster by the developer. A related feature is sending partial files, especially useful to send progressive JPEGs that show an initial low-res preview and then load completely after priority requests have completed. The effects of these two combined can give end-users an incredible perceived performance boost.

I assume that both these features would require support from the networking stack, is this on the roadmap?

Additionally, HTTP/2 support for Azure would be nice, and work on that seems to have started.

davidfowl commented 7 years ago

I assume that both these features would require support from the networking stack, is this on the roadmap?

HTTP/2 works today with WebListener and when using IIS on Windows 10/Server 2016 (though we don't yet expose the push promise API https://github.com/aspnet/HttpAbstractions/issues/371). Work is happening post 2.0 to support it natively in Kestrel which requires ALPN support from SSL Stream (which is mentioned above).

Drawaes commented 7 years ago

I think the Http client is interesting because out of the box it certainly isn't great for service to service communication (max 2 connections etc). But I think if the performance is right then a "wrapped" http client with better default settings would solve most of the problems people hit. It's more a guidance than a missing features problem.

Today I always wrap it for s2s in our environments and pass it to devs with settings tweaked.

bgever commented 7 years ago

@davidfowl Awesome, thanks for the quick update!

SteveDesmond-ca commented 7 years ago

It would be great to get some xplat consistency here -- I spent quite some time debugging why perf for one app was so different between OSes...turns out CurlHandler is quite a bit faster than WinHttpHandler!

karelz commented 7 years ago

Re: WCF

@ewinnington WCF technology is built on top of Networking stack. The WCF team was involved (@mconnew) in our reviews to make sure we are not dropping important features for WCF. Discussion about WCF client or server is outside of scope here. We think about Networking stack as silent (or at least minimal-work) replacement of current Networking APIs, so all upstack technologies will be able to use it right away (or with minimal effort in the worst case).

Re: HTTP/2

@bgever HTTP/2 is on our list (see Emerging Technologies in top post). We are also aware of ASP.NET HTTP/2 efforts and we plan to reuse code and effort as much as possible.

@stevedesmond-ca our technical roadmap is primarily motivated by perf and consistency across platforms. Perf is especially needed in HttpClient & SslStream for client and middle-ware scenarios.

Horusiath commented 7 years ago

@karelz Are there any plans to support transfer file API?

slang25 commented 7 years ago

I'd expect that the recommendation would be to continue to use a single shared instance of HttpClient where possible, would the DNS caching issue outlined below be fixed as a result of this new managed implementation:

stephentoub commented 7 years ago

Are there any plans to support transfer file API?

@Horusiath, can you elaborate? Are you asking for Socket.SendFile? Or APIs that wrap Windows' background file transfer APIs? Or something else?

stephentoub commented 7 years ago

would the DNS caching issue outlined below be fixed as a result of this new managed implementation

What change are you hoping for?

Suchiman commented 7 years ago

@stephentoub the summary of the links @slang25 and i posted:

The Primary concern is: can we remove this unexpected trap that seems to be both known and nothing done about while also being a luring trap many are not aware about.

Horusiath commented 7 years ago

@stephentoub I'm talking about the TransmitFile on Windows and sendfile on Linux. Both of them enable sending files from disk directly to socket using a single system call, without need of using user code in the middle.

stephentoub commented 7 years ago

I'm talking about the TransmitFile on Windows and sendfile on Linux

@Horusiath, that already exists: https://msdn.microsoft.com/en-us/library/system.net.sockets.socket.sendfile(v=vs.110).aspx It's implemented with TransmitFile on Windows: http://source.dot.net/#System.Net.Sockets/System/Net/Sockets/SocketPal.Windows.cs,211 and with sendfile on Unix: https://github.com/dotnet/corefx/blob/8c696bde9b27cd2d0de7540ccc2f4c11ac11491c/src/System.Net.Sockets/src/System/Net/Sockets/Socket.Unix.cs#L148 What is it that's missing?

stephentoub commented 7 years ago

HttpClient implements IDisposable... If you singleton the HttpClient to workaround this terrible behavior...

HttpClient was explicitly designed to be used as a singleton, or rather with as few HttpClient instances as possible, reusing them for all operations rather than creating a new one per operation. But it still represents resources, so it implements IDisposable for cleaning up after it if there's a time when you need to do that.

This is inherent to the design. Are you hoping to see that changed somehow, and if so, how? If your concern is around education, helping devs to better understand and employ that design, then I agree, but I'm not sure what could be done about that in the APIs themselves (I could imagine things done external to the APIs, e.g. better documentation, some kind of static analysis, etc.). If you have suggestions, please share.

On .NET Framework you can limit the impact of the DNS Cache by using the ServicePoint class which is unavailable on .NET Core AFAIK.

It's there in .NET Core 2.0, though this particular setting currently isn't respected by either the WinHttp or libcurl implementations, in part because (at least to my knowledge) those underlying providers don't have similar knobs. So, what exactly is the change you're hoping for? Are you asking for new APIs to be added to provide more fine-grained control over the connection pooling? Are you asking for an API to disable connection pooling? Are you asking for an API to timeout connections after some period of time? Are you just asking for the .NET Core implementation to respect ServicePointManager settings related to this? Etc. Since this managed implementation would be managing its own connection pooling, it will have more of an ability to provide fine-grained controls over things, and then the question is if/how such settings should be controllable through API, and such API-level requests should be in a separate issue.

Thanks.

Horusiath commented 7 years ago

@stephentoub Socket.SendFile exists only on .NET CLR. It doesn't appear in .NET Core.

stephentoub commented 7 years ago

Socket.SendFile exists only on .NET CLR. It doesn't appear in .NET Core.

It does. It was added for .NET Core 2.0.

davidfowl commented 7 years ago

HttpClient was explicitly designed to be used as a singleton, or rather with as few HttpClient instances as possible, reusing them for all operations rather than creating a new one per operation. But it still represents resources, so it implements IDisposable for cleaning up after it if there's a time when you need to do that.

You can draw a parallel with SqlConnection which is also IDisposable and uses a connection pool under the covers to get reuse but that doesn't bleed into user code. You're never told to keep the SqlConnection around as a singleton and not to dispose it. So one concrete change that could be made would be to drop whatever affinity we have with the HttpClient instance and use an underlying pool/state/store to cache things.

galvesribeiro commented 7 years ago

I agree with @davidfowl that having an under-the-hood pool just like on SqlConnection is the way to go. However, people need to be educated to ensure they understand how it will work just like they do with SqlConnection.

davidfowl commented 7 years ago

The EntityFramework team did something recently where you can configure a pool for DbContext instances. The usage of the DbContext itself didn't change but another layer of indirection was introduced into the system so disposing the DbContext would return it to the pool when running in this mode.

stephentoub commented 7 years ago

So one concrete change that could be made would be to drop whatever affinity we have with the HttpClient instance and use an underlying pool to cache things.

That could of course be done. But that alone is an incomplete solution... how does someone release the resources in the pool explicitly? How does someone express isolation of pools? Etc.

Let's please move any such discussion to the corefx repo. If someone would like to open an issue in corefx with a concrete proposal for what changes would be made, that'd be a great place to have a discussion about if/what changes could be made.

Drawaes commented 7 years ago

The problem with the SQL connection analogy is really the dns story. It is very rare to have dns provide failover there and often multiple servers are provided directly to the client. Having a magic pool could make the dns problem worse. (P.s. other frameworks have issues on DNS caching).

galvesribeiro commented 7 years ago

Hummm... make sense in that case @Drawaes... What would be your suggestion? Don't pool at all and make the user handle explicitly the lifecycle of the HttpClient (i.e. releasing resources on IDisposable.Dispose() and reseting DNS cache)?

IIRC, the DNS cache is something managed by the OS networking stack, right? If I'm right, I wonder how a fully managed client would bypass that...

davidfowl commented 7 years ago

Lets move this to an actual issue on corefx guys. @galvesribeiro Can you open an issue?

galvesribeiro commented 7 years ago

Let's go there: https://github.com/dotnet/corefx/issues/21568

clrjunkie commented 7 years ago

"better late than never..." :)) https://github.com/dotnet/corefx/issues/6771

clrjunkie commented 7 years ago

Issue: Do we need a server API other than ASP.NET? This would be a replacement for HttpListener, which is legacy.

Yes you do. Hopefully you would understand it much sooner...

mgravell commented 7 years ago

@karelz re "I personally view Pipelines and Span<T> as lower-layer primitives" - I agree to an extent, but to get most of the benefits, you do need to fundamentally architect to the strengths of pipelines; it certainly isn't a direct swap, so can't really be treated as an implementation detail of other APIs. But: appreciate the clarification that it remains a parallel conversation for now.

karelz commented 7 years ago

@mgravell agreed. That's why I said "We keep them in mind as we work on the new stack" -- it means we consider those technologies/ideas as part of our architecture. When we can design for future/flexibility for not-too-high cost, we do that. We just don't want to boil the entire ocean (yet ;)).

fyndor commented 6 years ago

I see HttpListener is considered legacy and it seems your answer server side is ASP.NET. What is ASP.NET using under the hood? I am assuming it is not HttpListener? Will that be exposed directly or is it too tightly coupled to ASP.NET?

Tratcher commented 6 years ago

ASP.NET Core exposes two servers for different scenarios, Kestrel and WebListener. WebListener is forked from HttpListener. Kestrel uses raw sockets. Neither are intended to be used outside ASP.NET.

What do you pan to build that requires using the server directly?

clrjunkie commented 6 years ago

What do you pan to build that requires using the server directly?

Answers to this question shape and justify the requirements on the server side of the "Web stack", therefore I believe it should be addressed to all.

I want the “server” because I want to replace HttpListener (which I have been using inside my application since .NET 2.0) with a high performance substitute having consistent API semantics across platforms.

I want the “server” in order to implement an RPC framework over HTTP similarly to Google’s gRPC

From: https://grpc.io/faq/ Why is gRPC better/worse than REST?

gRPC largely follows HTTP semantics over HTTP/2 but we explicitly allow for full-duplex streaming. We diverge from typical REST conventions as we use static paths for performance reasons during call dispatch as parsing call parameters from paths, query parameters and payload body adds latency and complexity. We have also formalized a set of errors that we believe are more directly applicable to API use cases than the HTTP status codes.

I want the “server” in order to expose an HTTP interface to my new custom developed server application be it my embedded database or a push notification server which has nothing to do with HTML or "Web Application Technologies"

I want the “server” in order to replace the WCF HttpBinding that uses HttpListener in the Full Framework with a new high performance cross-platform Http server, so the next time I think of developing a cache server similar to AppFabric Cache customers could easily access it using different programing languages and won’t say it’s SLOW.

I want the “server” to develop my own Web Proxy solution for filtering HTTP Traffic and I don’t want any ASP.NET code in the middle.

I want the “server” because I want to develop a Web Framework that’s BETTER than ASP.NET because I think I can and that’s how innovation fosters.

I want to develop a private protocol over HTTP and the last thing I care about is complying with HTTP standards where ASP.NET abstractions are required to follow, be it form data handling or what not.

clrjunkie commented 6 years ago

https://blog.cloudflare.com/exposing-go-on-the-internet/

Drawaes commented 6 years ago

I think you can do most of that. If you use Kestrel without MVC it is very close to the go http handling. Also if you look at the server and transport abstractions in 2.0 I am not sure what else you would want to hook or strip away yourself.

Kestrel doesnt do any form validation or handling that is done at the MVC level..

clrjunkie commented 6 years ago

@Drawaes

ASP.NET Core exposes two servers for different scenarios, Kestrel and WebListener. WebListener is forked from HttpListener. Kestrel uses raw sockets. Neither are intended to be used outside ASP.NET

davidfowl commented 6 years ago

@clrjunkie I think exposing a low level server from .NET Core is fine but it's not required to do any of the things you mentioned. While I don't want to force ASP.NET Core on anybody that doesn't need it, those examples you called out aren't compelling reasons to write and maintain another server.

I want to develop a private protocol over HTTP and the last thing I care about is complying with HTTP standards where ASP.NET abstractions are required to follow, be it form data handling or what not.

I don't know what this means. Do you mean you want to use HTTP headers and a different http body format? What exactly are you talking about?

That said, as we further refactor kestrel, we'll end up with very low level components that you can use to implement your own http server without the ASP.NET HttpAbstractions. That may or may not evolve into something that you want, but it's on the roadmap.

clrjunkie commented 6 years ago

@davidfowl

I’m very puzzled as to what in my post above suggested that I advocate for another server or that Kestrel doesn’t fill in the needs .

My post was in response to this:

ASP.NET Core exposes two servers for different scenarios, Kestrel and WebListener. WebListener is forked from HttpListener. Kestrel uses raw sockets. Neither are intended to be used outside ASP.NET. What do you pan to build that requires using the server directly?

As far as I’m concerned, the question is equivalent to:

“What do you pan to build that requires using the Kestrel or HttpListener directly?”

Kestrel may very well be ok if you guys intend it be used outside of ASP.NET

Having said that,

1) I think a “Kestrel Http Server” outside CoreFx is an awkward decision. If it’s so “generic” why aren’t you including it in the standard library, like Go does? IMHO, this whole “Kestrel” branding is so irrelevant for what obviously needs to be part of the standard class-library.

2) As I wrote in a separate issue, you need to provide an adapter class that converts between Kestrel API and Http Listener API , so people can benefit from a SINGLE Http server implementation. Your whole “Http Listener legacy” story is nothing I would even consider using.

I don't know what this means. Do you mean you want to use HTTP headers and a different http body format? What exactly are you talking about?

Yes. I don't want anything besides Http Header parsing.

omariom commented 6 years ago

What I'd like to see is building blocks used by other buildings blocks and all these building blocks used to build servers (including Kestrel). Abstractions are not free and the perf can suffer but at least it should be tried :)

clrjunkie commented 6 years ago

Reconsider port to core approach for System.Net.HttpListener https://github.com/dotnet/corefx/issues/13610