dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.27k stars 4.73k forks source link

Updating to .net 6 causes higher lock contention #108057

Closed AlanLiu90 closed 1 month ago

AlanLiu90 commented 1 month ago

Description

Recently, I upgraded our game server project to .net 6 from .net core 3.1 and observed Monitor.LockContentionCount became significantly higher during stress test. Using dotnet-trace, I found it's from Thread.SetThreadPoolWorkerThreadName

image-20240920114038998 Looking into the source code, it seems the contention is from the following code:

private static void CreateWorkerThread()
{
    // Thread pool threads must start in the default execution context without transferring the context, so
    // using UnsafeStart() instead of Start()
    Thread workerThread = new Thread(s_workerThreadStart);
    workerThread.IsThreadPoolThread = true;
    workerThread.IsBackground = true;
    // thread name will be set in thread proc
    workerThread.UnsafeStart();
}

private unsafe void StartCore()
{
    lock (this)
    {
        fixed (char* pThreadName = _name)
        {
            StartInternal(GetNativeHandle(), _startHelper?._maxStackSize ?? 0, _priority, pThreadName);
        }
    }
}

private static void WorkerThreadStart()
{
    Thread.CurrentThread.SetThreadPoolWorkerThreadName();

    // ...
}

internal void SetThreadPoolWorkerThreadName()
{
    Debug.Assert(this == CurrentThread);
    Debug.Assert(IsThreadPoolThread);

    lock (this)
    {
        _name = ThreadPool.WorkerThreadName;
        ThreadNameChanged(ThreadPool.WorkerThreadName);
    }
}

Although I haven't seen performance problem yet, but I think it causes some confusion when updating to .net 6 or later versions.

It looks like using SetThreadPoolWorkerThreadName instead of setting Name property is to avoid setting _mayNeedResetForThreadPool. Is it better to modifiy the implementation to avoid this lock contention?

Reproduction

  1. dotnet new console
  2. Modify TargetFramework to net6.0 (Also reproducible in net8.0)
  3. Paste the following code to Program.cs
    
    using System;
    using System.Linq;
    using System.Threading;

namespace ConsoleApp1 { internal class Program { private static double[] mValues;

    static void Main(string[] args)
    {
        StartBusyThreads();

        long oldCount = Monitor.LockContentionCount;

        ManualResetEvent done = new ManualResetEvent(initialState: false);

        const int MaxCount = 1000;
        int count = 0;

        for (int i = 0; i < MaxCount; i++)
        {
            ThreadPool.QueueUserWorkItem(m =>
            {
                Thread.Sleep(1000);

                if (Interlocked.Increment(ref count) == MaxCount)
                {
                    done.Set();
                }
            });
        }

        done.WaitOne(TimeSpan.FromSeconds(10));

        Console.WriteLine("LockContentionCount: {0}", Monitor.LockContentionCount - oldCount);
        Console.WriteLine("DummyValue: {0}", mValues.Sum());

        Environment.Exit(0);
    }

    static void StartBusyThreads()
    {
        var values = mValues = new double[Environment.ProcessorCount];
        var threads = new Thread[Environment.ProcessorCount];

        using var sem = new Semaphore(0, Environment.ProcessorCount);

        for (int i = 0; i < Environment.ProcessorCount; i++)
        {
            int index = i;

            var t = new Thread(_ =>
            {
                sem.Release();

                while (true)
                {
                    const int Count = 10000;

                    double t = 0;

                    for (int j = 0; j < Count; j++)
                        t += Math.Sin(j / (double)Count) + Math.Cos(j / (double)Count);

                    values[index] += t;
                }
            });

            t.Start();
            threads[i] = t;
        }

        for (int i = 0; i < Environment.ProcessorCount; i++)
        {
            sem.WaitOne();
        }
    }
}

}

4. build
5. Use dotnet-trace to run the application:

dotnet-trace collect --clrevents contention+stack --clreventlevel informational --duration 00:00:10 -- ConsoleApp1.exe

martincostello commented 1 month ago

Do you see the same with .NET 8? .NET 6 goes out of support in two months.

AlanLiu90 commented 1 month ago

Do you see the same with .NET 8? .NET 6 goes out of support in two months.

Yes, .NET 8 has the same issue.