search

dotnet / aspnetcore

ASP.NET Core is a cross-platform .NET framework for building modern cloud-based web applications on Windows, Mac, or Linux.
https://asp.net
MIT License
35.35k stars 9.99k forks source link

.Net Core app throughput degrades over the time and increases latency in VS Load Test #20774

Closed vlkumarster closed 4 years ago

vlkumarster commented 4 years ago

Describe the bug

Created .NET app which writes to Storage and keeps the reference of it in a Service Bus. We have enabled Request and Response logging into AppSights. The app works without issues and return result in <250ms normally, but if we run the perf test for 30 min with 10 users with gradual increase in load, the latency increases and throughput decreases. Can you let us know what could be the issue here? We are hosting this in Kubernetes using Ingress and HPA.

perfissue

To Reproduce

at System.Net.HttpWebRequest.EndGetResponse(IAsyncResult asyncResult) at Microsoft.VisualStudio.TestTools.WebStress.WebTestTransaction.ResponseReceived(IAsyncResult result) ### Further technical details - ASP.NET Core version : 3.1 - Include the output of `dotnet --info` - The IDE (VS / VS Code/ VS4Mac) you're running on, and it's version : VS 2019
mkArtakMSFT commented 4 years ago

Hi. It looks like you are looking for support for your application, which this forum is not intended for. If I have misunderstood you and you believe this is caused by an issue in the ASP.NET Core framework, please share a sample app with the minimum additions, which can reliably reproduce the unexpected behavior. If, however, you are indeed looking for support, we recommend using StackOverflow as you will get faster responses in there.

dewanymca commented 4 years ago

Thanks @mkArtakMSFT . Actually we are facing issue with scaling of app, everything works fine under normal load and even with 10 user load app works fine. We are seeing issue whenever we are increasing load on app by increasing user or start test with 50 user itself. Below are couple of things we tried but cou;dn't resolve it:

1) Disable the middleware which logs request and response to AI 2) Disable the Console Logging and keeping it to Error Level only,

What we have observe, app works great in initial 5-10 minute(able to process 10k request) and after this become sluggish and take 20 minute to process same set of 10k request. Though scaling happened at AKS layer to handle more traffic in term of POD. All of this is getting correlated with increase in latency during sluggishness. We checked app telemetry and our dependency(storage and servicebus) telemetry are working good in terms of performance.

davidfowl commented 4 years ago

Maybe capture profiles of your application. Get dotnet-trace and dotnet-counters. The counters should give you a high level view of your application stats. If you’re using app insights it should have those same counters in the portal metrics blade

ghost commented 4 years ago

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment. If it is closed, feel free to comment when you are able to provide the additional information and we will re-investigate.

See our Issue Management Policies for more information.

ghost commented 4 years ago

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment. If it is closed, feel free to comment when you are able to provide the additional information and we will re-investigate.

See our Issue Management Policies for more information.