Macro buttons with images appear to trigger memory leak in macro editor, other macro management task performance issues

[RicoTheBold]

Describe the bug Under specific circumstances, opening the macro editor and saving a changed macro will consume a disproportionately high level of memory that is never released. These circumstances appear to involve many macros with asset IDs referenced in their names. It also appears that performance of several macro-manipulation actions (saving, clearing groups) is also impaired.

To Reproduce Steps to reproduce the behavior:

• Optionally: Clear asset/thumbnail cache (I did it to make sure this would replicate well with this campaign file on other systems)
  1. Open a fresh instance of Maptool
  2. Load test campaign
  3. Open GM macro panel
  4. Right click a macro button, choose to edit the file (memory usage will jump some)
  5. Click the Options tab, and check or uncheck the "Allow Players to Edit Macro" button
  6. Hit OK (memory usage will jump a bunch more)
  7. Repeat steps 4-7 on another macro until you get bored or run out of heap space.
  8. Optionally: Attempt to reclaim memory by double-clicking the memory status bar. You generally can reclaim roughly the amount increased the last time you did step 4.

(other performance issues are noted below)

To not reproduce (control/negative test) On step 3, choose the Campaign panel. Follow steps 4-7.

Other notes - references to panels are specific to this test campaign file

• You can generally reclaim some memory by double-clicking the memory bar, but it seems to be about the amount for opening just the last macro, not the additional memory usage after saving one.
• It looks like there's some memory cleanup that's otherwise working - I can get the memory to balloon up some on an empty campaign file (or the campaign panel on this one), but it regularly drops right back down to about where it started...or I can double-click to get back to that level. Some evidence of that is noted below in the screenshots of tests with the Campaign panel's macros.
• It's not inherent to the GM panel - most of these macros have been on my campaign panel for many versions of Maptool in the 1.5 era, and the problem has been there for some time. It's actually the introduction of the GM panel, and my using it as a place to start writing some new macros, that made me notice I wasn't having the problem there.
• The macros on the campaign panel are all super simple ones from an old Savage Worlds framework I had - they just toggle some states (non-existent in this file).
• Most of the macros on the GM panel are actually comparably simple - lots of them are just single-line macros that call macros from lib tokens (non-existent in this file).
• As a result, I don't think it's the length of the macros themselves.
• There's a test macro with an image on the campaign panel that doesn't have the problem...unless you move it to the GM panel.
• The macros on the GM panel without images also cause memory to spike, so it doesn't seem to be tied to the individual macro.
• Saving the macros on the GM panel (after opening them) is much slower than the ones on the campaign panel.
• If you try the "Clear Group" option (from a right-click) on the GM panel, it is also very slow (freezing Maptool for the duration while it happens). It took 1 minute and 47 seconds to delete the 3-macro group 220-Doors. Clearing a group from the campaign panel is nearly instantaneous. The performance of the clear group function improves as more and more groups are removed from the GM panel.
• Deleting an individual macro (or duplicating one) doesn't seem any slower with the affected macros.
• If you try the "Clear Panel" option (from a right-click) it is nearly instant on both panels.
• I think the performance/memory hit also happens when the macros are on a token instead of on a panel, but I haven't tested it since I made the connection that the asset images && the number of macros could allow the issue to be replicated on an otherwise empty campaign. Mostly I've encountered the issue when I'm just making a bunch of modifications at a time, so I'm almost always trying to actually do something and it's historically been hard to stop and switch to troubleshoot mode.

Expected behavior Opening and saving a one-line macro should not 100-200MB of memory when the entire campaign file is < 3 MB. Slowdown should not occur in macro-saving options or group-clearing options when macros have images in the name.

Screenshots

Test of issue (GM panel macros) Cleared caches to start:

Memory after loading campaign:

Memory after editing 6 macros on the GM panel:

Memory after editing 17 macros on the GM panel:

Memory after editing 27 macros on the GM panel:

Memory after editing 30 macros on the GM panel (I stopped here because my heap space was set to 6 GB, and it absolutely will give you heap errors if you continue):

Memory after double-clicking the memory bar (didn't seem to reclaim any this time - possibly too close to the heap space maximum...): (Oops, apparently I overwrote this screenshot. Basically it looked unchanged, but usually I can reclaim around .15 to .3 GB)

Control/Negative test (Campaign panel macros) Immediately after restarting and loading the campaign to test using the macros on the campaign panel

Memory after editing 15 campaign panel macros:

(Note that this has less memory used than immediately after load. This is accurate to what happened during the test. Memory usage seems to jump up some at times, particularly while loading/saving macros, but occasionally get cleared out in a way that doesn't seem to happen in the other test. I'm guessing there's a garbage collection process that runs intermittently and this just happened to fall in that cycle...it periodically jumped up/down to about the previous level again between the 15 and 30 macros tests.)

Memory after editing 30 campaign panel macros:

I believe I was able to drop the memory usage back to something like 700MB again after double-clicking my bar post-30 campaign macro edits, but I didn't save a file with that name so I can't demonstrate it.

MapTool Info

• Version: 1.5.8
• Install: Upgrade (from many previous versions)

Desktop (please complete the following information):

• OS: Windows
• Version 10, 1903

Additional context Test Campaign file: 1memtest.zip

[RicoTheBold]

Forgot to mention:

• Issue is not new to 1.5.8. I've had the issue with my main campaign file for some time, possibly even back to the Nerps fork versions, but it was actually while using the new GM panel as a convenient test ground while writing new macros (without asset ID names) and having no performance issues/memory jumps that gave me the couple of clues I needed to start narrowing this down in a testable way.
• Basically any save operation will trigger the continued memory jumps. Changing the player editable flag is just an easily-replicated example for the test I ran (and not coincidentally, what I was doing when I got annoyed into figuring out why it was so terrible on one panel and not the other in my main campaign file I'm prepping to clean up into a releasable framework).

[Phergus]

This happens on both the GM, Selected and Campaign macro panels with the memory being lost for each image button macro opened in the editor.

Clearing the panel does not recover the memory nor does selecting New Campaign from the file menu.

Opening the same macro in the editor and clicking on OK multiple times has the same effect as editing multiple different macros.

The amount lost each time may be related to the image size and/or number of image button macros on the panel.

[pizzystrizzy]

I believe this is related to the fact that for tokens containing many macro buttons with images, there is a significant lag between the time you click and the time they are actually selected.