jet / propulsion

.NET event stream projection and scheduling platform with CosmosDB, DynamoDB, EventStoreDB, MemoryStore, message-db, Equinox and Kafka integrations
https://github.com/jet/dotnet-templates
Apache License 2.0
177 stars 24 forks source link

Solve memory leak caused by `Task.WhenAny` usage #217

Closed nordfjord closed 1 year ago

nordfjord commented 1 year ago

Fixes leak per cycle in:

Before (5.83MB in Gen2 at 1m):

Screenshot 2023-06-10 at 23 26 08

After (3MB in Gen2 at 1m and less memory pressure over all):

Screenshot 2023-06-10 at 23 27 32

See the test code I ran to get these profiles here

Task.WhenAny doesn't cancel the rest of the Tasks when one succeeds, therefore we must create a linked cancellation token to cancel the stuff we don't want to leak.

bartelink commented 1 year ago

Great work, thank you! If only I'd have managed to infer all Stephen Cleary was implying in https://stackoverflow.com/questions/73233208/using-task-whenany-to-await-capacity-on-a-semaphoreslim/73233209#73233209 at the time and been able to get it right in the first instance... Ping me when it's ready to merge and I'll get it out the door ASAP (will also ship https://github.com/jet/equinox/pull/394 at the same time as there are (not) static types I need to Sealed; AbstractClass)