search

KirillOsenkov / MSBuildStructuredLog

A logger for MSBuild that records a structured representation of executed targets, tasks, property and item values.
MIT License
1.41k stars 188 forks source link

Unbounded memory consumption when viewing log files causes system unresponsive / crash #783

Closed jhennessey closed 1 week ago

jhennessey commented 3 weeks ago

Opening log files is consistently causing unbounded memory consumption which eventually leads to an unresponsive system / machine crash.

Most recently this occurred with a log file that was ~ 130MB in size. The process consumed over 14GB in memory before it was killed (letting it run will eventually cause unresponsiveness / machine crash). This has happened on multiple occasions and makes the application unusable for troubleshooting purposes.

Note that the Enable tree virtualization option (along with all others) was not enabled and this occurred using version 2.2.248. The log contains sensitive data so I can't share it. But I'm happy to troubleshoot or provide any additional details that may be useful.

Thanks for all the work on this application!

KirillOsenkov commented 3 weeks ago

Unfortunately I don't think I'll be able to do anything about this, sorry.

jhennessey commented 2 weeks ago

Unfortunately I don't think I'll be able to do anything about this, sorry.

No worries. In your experience, is this size log file just too large for the viewer to handle?

KirillOsenkov commented 2 weeks ago

yes, logs that are 300MB in size need 64GB RAM to open. The format is highly compressed and expands to consume gigabytes. When it does open it prints the stats about memory at the bottom. If you want you can send me your binlog privately at my first name dot last name at microsoft com and I can see if there are ways to make it smaller. You can use File -> Statistics to see what takes up space.

KirillOsenkov commented 2 weeks ago

how much RAM do you have on your machine? do you have a machine with >= 64 GB RAM?

jhennessey commented 2 weeks ago

I was able to open it on a machine with 64 GB :). The previous machine only had 16GB and the process was at ~30 GB after opening.

Here are the statistics (if interested): image

Thanks!