search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.25k stars 4.73k forks source link

How to deal with traffic bursts #107013

Open zhenlei520 opened 2 months ago

zhenlei520 commented 2 months ago

Description

  1. Simulate 100 connections, accessing at an average of 50 qps per second
  2. Simulate response delays to cause thread pool threads to accumulate
  3. Response time becomes longer, thread creation takes time, a large number of requests accumulate, and CPU usage increases sharply

Configuration

System: Ubuntu 22.04 X64 .NET Version: 8.0

Data

image

Analysis

Downstream.zip

The response time is longer, and the number of requests remains the same, which means more requests need to be processed, and more threads are needed to process these requests, so the CPU increases. However, my setting of MinThreads has no effect at all. The thread increase still takes 1 minute. What I hope is that a certain amount of threads can be prepared in advance so that when burst traffic comes in, these threads can be used directly without temporarily enabling new threads. Temporarily enabling a large number of requests will cause a lot of CPU pressure and easily cause the server to go down.

ThreadPool.SetMinThreads(200, 200);

ThreadPool.SetMaxThreads(1000, 1000);
dotnet-policy-service[bot] commented 2 months ago

Tagging subscribers to this area: @mangod9 See info in area-owners.md if you want to be subscribed.

zhenlei520 commented 2 months ago

https://github.com/microsoft/reverse-proxy/issues/2427#issuecomment-1977911480

We use it in yarp and find that a lot of CPU resources are occupied in SetThreadPoolWorkerThreadName, but this is a runtime problem. Is there any room for optimization?