rnwood / smtp4dev

smtp4dev - the fake smtp email server for development and testing
BSD 3-Clause "New" or "Revised" License
2.94k stars 334 forks source link

Memory leak in newer versions #1479

Open ryan-adler opened 1 month ago

ryan-adler commented 1 month ago

Until recently, we were running the smtp4dev container without any issues on version linux-amd64-3.2.0-ci20221023104. We have moved to a newer container with the latest tag (3.6.0+32d49cac3a0c67e55228cd39ca66d3b552afa7d6), and we see a constant increase in memory usage until we get OutOfMemoryExceptions.

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown.
   at Microsoft.Data.Sqlite.SqliteValueReader.GetBlob(Int32 ordinal)
   at Microsoft.Data.Sqlite.SqliteValueReader.GetFieldValue[T](Int32 ordinal)
   at lambda_method110(Closure, QueryContext, DbDataReader, ResultContext, SingleQueryResultCoordinator)
   at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable`1.Enumerator.MoveNext()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
   at Rnwood.Smtp4dev.Controllers.MessagesController.GetSummaries(String searchTerms, String mailboxName, String sortColumn, Boolean sortIsDescending, Int32 page, Int32 pageSize) in /home/vsts/work/1/s/Rnwood.Smtp4dev/Controllers/MessagesController.cs:line 81
   at lambda_method98(Closure, Object, Object[])
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerAct

image

rnwood commented 1 month ago

Thanks for reporting this.

Are you running with DB as a file (the default) as in memory? What kind of volume of sessions/messages before you are hitting this?


From: ryan-adler @.> Sent: Wednesday, May 15, 2024 7:45:39 AM To: rnwood/smtp4dev @.> Cc: Subscribed @.***> Subject: [rnwood/smtp4dev] Memory leak in newer versions (Issue #1479)

Until recently, we were running the smtp4dev container without any issues on version linux-amd64-3.2.0-ci20221023104. We have moved to a newer container with the latest tag, and we see a constant increase in memory usage until we get OutOfMemoryExceptions.

System.OutOfMemoryException: Exception of type 'System.OutOfMemoryException' was thrown. at Microsoft.Data.Sqlite.SqliteValueReader.GetBlob(Int32 ordinal) at Microsoft.Data.Sqlite.SqliteValueReader.GetFieldValue[T](Int32 ordinal) at lambda_method110(Closure, QueryContext, DbDataReader, ResultContext, SingleQueryResultCoordinator) at Microsoft.EntityFrameworkCore.Query.Internal.SingleQueryingEnumerable1.Enumerator.MoveNext() at System.Collections.Generic.List1..ctor(IEnumerable1 collection) at System.Linq.Enumerable.ToList[TSource](IEnumerable1 source) at Rnwood.Smtp4dev.Controllers.MessagesController.GetSummaries(String searchTerms, String mailboxName, String sortColumn, Boolean sortIsDescending, Int32 page, Int32 pageSize) in /home/vsts/work/1/s/Rnwood.Smtp4dev/Controllers/MessagesController.cs:line 81 at lambda_method98(Closure, Object, Object[]) at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerAct

image.png (view on web)https://github.com/rnwood/smtp4dev/assets/109764269/4400c5ea-71bf-40b0-b307-f342266dd740

— Reply to this email directly, view it on GitHubhttps://github.com/rnwood/smtp4dev/issues/1479, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AAKEGF4RJYQFCEVYARPAEETZCMABHAVCNFSM6AAAAABHXQID6OVHI2DSMVQWIX3LMV43ASLTON2WKOZSGI4TOMBQGQYDSNI. You are receiving this because you are subscribed to this thread.Message ID: @.***>

ryan-adler commented 1 month ago

It is running from the container, with basically default settings, except for enabling StartTls, NumberOfMessageToKeep at 5000,, setting the hostname, port, and locking settings. So I guess the DB is using the default file mode. I would guess around 500-1000 mails is the expected volume. I can give a more accurate estimate tomorrow after our test suites have run against the old container and I can look at the history there.

rnwood commented 1 month ago

Please can you confirm if this is reproducable for you with >= 3.6.1.

I'm not able to reproduce this with 3.6.0, however the stress testing script I have and settings may not match yours. In particular, mine currently sends a large number of messages per session.

ryan-adler commented 1 month ago

Please give us some time - we need to have the devs move tests to the new API before we can fully test. I'll let you know as soon as possible.