Processor issues in Docker/Nginx environment using Websocket middleware

[aspnet-hello]

From @villecoder on Wednesday, June 21, 2017 11:37:39 AM

I'm following Radu Matei's implementation of WebSocket middleware for an ASP.NET Core website running behind docker.

Here are my project dependencies: "dependencies": { "Microsoft.NETCore.App": { "version": "1.1.0", "type": "platform" }, "Microsoft.AspNetCore.AngularServices": "1.1.0-*", "Microsoft.AspNetCore.Diagnostics": "1.1.0", "Microsoft.AspNetCore.Mvc": "1.1.0", "Microsoft.AspNetCore.Razor.Tools": { "version": "1.0.0-preview2-final", "type": "build" }, "Microsoft.AspNetCore.Server.IISIntegration": "1.1.0", "Microsoft.AspNetCore.Server.Kestrel": "1.1.0", "Microsoft.AspNetCore.StaticFiles": "1.1.0", "Microsoft.AspNetCore.WebSockets": "1.0.2", "Microsoft.Extensions.Configuration.EnvironmentVariables": "1.1.0", "Microsoft.Extensions.Configuration.Json": "1.1.0", "Microsoft.Extensions.Configuration.CommandLine": "1.1.0", "Microsoft.Extensions.Logging": "1.1.0", "Microsoft.Extensions.Logging.Console": "1.1.0", "Microsoft.Extensions.Logging.Debug": "1.1.0", "Microsoft.Extensions.Options.ConfigurationExtensions": "1.1.0", "Tigr8.SystemLib": "0.7.6.4", "Tigr8.CommonLib": "0.7.6.4", "System.Linq": "4.3.0", "Microsoft.AspNet.WebApi.Client": "5.2.2", "Microsoft.AspNet.Http.Extensions": "1.0.0-rc1-final", "Microsoft.AspNetCore.Mvc.WebApiCompatShim": "1.1.0" },

When we run it and connect a few clients, the processor begins as normal and slowly begins climbing higher. Eventually, docker reports that the container is using 100% of all cores (so an 8-core machine reports 800% CPU usage after 30 minutes or so).

Is there something in Radu's setup that causes this? Am I using the library right? Or is there an issue with the version of the libraries that I'm using that causes this performance issue?

Copied from original issue: aspnet/WebSockets#188

[aspnet-hello]

From @Tratcher on Wednesday, June 21, 2017 11:40:45 AM

Reference link?

[aspnet-hello]

From @davidfowl on Wednesday, June 21, 2017 11:42:26 AM

https://github.com/radu-matei/websocket-manager

@villecoder Did you try filing a bug on that repository? Have you profiled the application?

[aspnet-hello]

From @villecoder on Thursday, June 22, 2017 5:34:21 AM

@davidfowl I did not file a bug in that repository, but I intend to. I didn't see anything overtly counter-productive in their code, however. From what I've implemented, it's the standard implementation of OnConnected, OnDisconnected, and ReceiveAsync that you see in every example on how to create a websocket server with this library.

The reason I'm opening the issue is to discover if you have found any similar issues while running under docker within a similar environment. I saw kestrel had similar issues (https://github.com/aspnet/KestrelHttpServer/issues/1182#issuecomment-256799995) and was wondering if anyone has observed anything similar with this project.

[aspnet-hello]

From @davidfowl on Thursday, June 22, 2017 5:57:22 AM

Nope. The next step is to profile and see where the CPU time is spent

[mcatanzariti]

Hi,

Same issue here on AWS ECS Fargate without Ngnix. CPU and memory increase slowly during 3 or 4 days to finally reach 100% CPU and memory.

The application is a notification service, it does not receive anything, only sends json notifications. The load is around 2000 CCU on 2 AWS instances with 2 cores and 4 Gb each.

The notification service writes around 1 message per 10 minutes per connection. It means less than 10 messages per second for all the connections

[davidfowl]

@mcatanzariti have you taken a memory dump or a CPU profile? What does your code look like? That's the first step to investigating a performance issue like this...

[analogrelay]

Closing this as we haven't heard from you and generally close issues with no response after ~7 days. Please feel free to comment if you're able to get the information we're looking for and we can reopen the issue to investigate further!