ApiController: memory leak when returning large string arrays

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the bug

When returning a large string in my ApiController, the process memory goes up. But even after the garbage collector was triggered (manually or automatically), the memory doesn't fall back like you expect.

A small code example is listed below. The full minimal project is linked further down.

[HttpGet("memory")]
[OutputCache(Duration = 0, NoStore = true)]
public IEnumerable<string> Memory()
{
    List<string> list = new();

    for (var i = 0; i < 1_000_000; i++)
    {
        list.Add(i.ToString());
    }

   return list;
}

Expected Behavior

I expect the memory to fall back. When I also create a list in a small console application, the memory falls back like expected. This console project is listed below and in the reproduce repo I linked.

while (true)
{
    GC.Collect();
    GC.WaitForPendingFinalizers();
    Console.ReadLine();

    var list = GetList();
    Console.WriteLine(list.Count);
}
List<string> GetList()
{
    List<string> list = new();
    for (int i = 0; i < 1_000_000; i++)
    {
        list.Add(i.ToString());
    }

    return list;
}

Steps To Reproduce

I created a minimalistic project in the following repository: https://github.com/Paultje52/aspnet-string-array-memory-issue

This repository consists of two project: WebApplication with the issue in an ApiController and ConsoleApp with the expected behaviour in a console application.

Go to the page http://localhost:5100 to get the list and trigger the memory issue. After making that request multiple times, go to http://localhost:5100/collect to manually trigger the garbage collector.

Exceptions (if any)

No response

.NET Version

8.0.202

Anything else?

ASP.NET Core: 8.0.3
IDE: Visual Studio 17.9.5

Dotnet info

``` .NET SDK: Version: 8.0.202 Commit: 25674bb2f4 Workload version: 8.0.200-manifests.8cf8de6d Runtime Environment: OS Name: Windows OS Version: 10.0.22631 OS Platform: Windows RID: win-x64 Base Path: C:\Program Files\dotnet\sdk\8.0.202\ .NET workloads installed: There are no installed workloads to display. Host: Version: 8.0.3 Architecture: x64 Commit: 9f4b1f5d66 .NET SDKs installed: 8.0.202 [C:\Program Files\dotnet\sdk] .NET runtimes installed: Microsoft.AspNetCore.App 6.0.28 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 7.0.17 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.AspNetCore.App 8.0.3 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App] Microsoft.NETCore.App 6.0.28 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 7.0.17 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.NETCore.App 8.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App] Microsoft.WindowsDesktop.App 6.0.28 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 7.0.17 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Microsoft.WindowsDesktop.App 8.0.3 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App] Other architectures found: x86 [C:\Program Files (x86)\dotnet] registered at [HKLM\SOFTWARE\dotnet\Setup\InstalledVersions\x86\InstallLocation] Environment variables: Not set global.json file: Not found Learn more: https://aka.ms/dotnet/info Download .NET: https://aka.ms/dotnet/download ```

Hello Paul,

I see you're running into memory issues, but the problem isn't so much a "leak" as it is the massive list you're creating and returning directly from the API. When you generate a list with 1 million strings and return it, it will naturally consume a large amount of memory.

Even though you're manually triggering garbage collection, it won’t immediately free up memory as much as you’d expect, especially in a web environment like ASP.NET Core. The framework manages memory differently compared to a console app. In a console application, garbage collection is often more straightforward because it typically operates on a smaller scale, often in single-threaded environments. ASP.NET Core, on the other hand, manages multiple concurrent requests, connections, and other resources, which can cause memory to be held longer.

Instead of returning such a huge list, I’d suggest either pre-allocating an array and reusing it or breaking the data into smaller chunks (pagination or streaming). Web applications are not optimized for sending enormous amounts of data in one go.

Cheers

dotnet / aspnetcore