search

spaghettidba / WorkloadTools

A collection of tools to collect, analyze and replay SQL Server workloads, on premises and in the cloud
MIT License
234 stars 53 forks source link

Investigate massive memory leak #17

Closed spaghettidba closed 5 years ago

spaghettidba commented 6 years ago

When running for 15 minutes or more, it ends up using a lot of memory. Looks like Strings are the most allocated objects (sql statements).

spaghettidba commented 6 years ago

Reduced memory usage by discarding already saved normalized queries and keeping only the Int64 hash. See commit 547463aa9a3c793bee927a8956d8a2f19bdc4719

Might not be enough for the cases where the queries are extremely volatile. One factor that might be driving the volatility is TVPs, where the number of values captured in the incoming workload is extremely varying and leads to masive INSERT...VALUES statements of different sizes, when it should be recording only the call to the TVP instead.

spaghettidba commented 5 years ago

The memory leak was due to events queuing up in the listener, so I implemented a cache on disk to reduce memory consumption