dotnet / MQTTnet

MQTTnet is a high performance .NET library for MQTT based communication. It provides a MQTT client and a MQTT server (broker). The implementation is based on the documentation from http://mqtt.org/.
MIT License
4.5k stars 1.07k forks source link

ASP net apps consuming data use a lot more memory than console apps #2112

Open q-bertsuit opened 2 days ago

q-bertsuit commented 2 days ago

I've noticed that ASP net apps that consume MQTT messages also use a lot more memory than console apps that subscribe to the same data. The memory never seems to be released. Running dotMemory shows that the app uses a lot of unmanaged memory

Which component is your bug related to?

Memory consumption of the Producer console app, Consumer console app and Consumer Asp Net 6 app:

Image

Create a console app that pushes data to the bus:

Program.cs:

using System.Text;
using System.Text.Json;
using MQTTnet;
using MQTTnet.Client;

var factory = new MqttFactory();
var client = factory.CreateMqttClient();

var options = new MqttClientOptionsBuilder()
    .WithClientId("SenderClient")
    .WithTcpServer("localhost", 1883) 
    .Build();

await client.ConnectAsync(options, CancellationToken.None);

var dto = new LargeDto
{
    Id = 1,
    Name = "Sample",
    Data = new byte[1024 * 50]
};

var jsonData = JsonSerializer.Serialize(dto);
var message = new MqttApplicationMessageBuilder()
    .WithTopic("large/dto")
    .WithPayload(Encoding.UTF8.GetBytes(jsonData))
    .WithQualityOfServiceLevel(MQTTnet.Protocol.MqttQualityOfServiceLevel.AtMostOnce)
    .Build();

while (true)
{
    await client.PublishAsync(message, CancellationToken.None);
    await Task.Delay(1); 
}

await client.DisconnectAsync();

public class LargeDto
{
    public int Id { get; set; }
    public string Name { get; set; }
    public byte[] Data { get; set; }
}

Create an ASP Net web API (tested both .Net 6 and .Net 8)

Program.cs:

using WebApplication1;

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddHostedService<MqttHostedService>();
var app = builder.Build();

app.MapGet("/", () => "Hello World!");

app.Run();

MqttHostedService.cs

using MQTTnet;
using MQTTnet.Client;

namespace WebApplication1;

public class MqttHostedService : IHostedService
{
    private readonly IMqttClient _client;
    private readonly MqttClientOptions _options;

    public MqttHostedService()
    {
        var factory = new MqttFactory();
        _client = factory.CreateMqttClient();

        _options = new MqttClientOptionsBuilder()
            .WithClientId("ListenerClientAsp6")
            .WithTcpServer("localhost", 1883) 
            .Build();

        _client.ApplicationMessageReceivedAsync += e =>
        {
            return Task.CompletedTask;
        };
    }

    public async Task StartAsync(CancellationToken cancellationToken)
    {
        await _client.ConnectAsync(_options, cancellationToken);
        await _client.SubscribeAsync(new MqttTopicFilterBuilder()
            .WithTopic("large/dto")
            .Build());
    }

    public async Task StopAsync(CancellationToken cancellationToken)
    {
        var disconnectOptions = new MqttClientDisconnectOptions
        {
            Reason = MqttClientDisconnectOptionsReason.NormalDisconnection,
            ReasonString = "Service stopping"
        };

        await _client.DisconnectAsync(disconnectOptions, cancellationToken);
    }
}
xljiulang commented 1 day ago

This is not a bug, it's a different GC mode. https://learn.microsoft.com/en-us/dotnet/core/runtime-config/garbage-collector#workstation-vs-server