Very severe memory leak when CompactGUI is left running overnight.

[ValiantBlade]

Exactly what the title says, I failed to capture a screenshot like an idiot, but upon waking up for the morning, I checked my task manager because I've been trying to track down a few long term memory leaks, and CompactGUI was using roughly a gig of RAM doing nothing, no analysis was active or recompression tasks at the time of checking.

[Iridium-IO]

I tried leaving it running overnight but it was only using 45MB the next morning.

Can you paste the contents of your watchedfolders.json file from the %IridiumIO%/CompactGUI folder here?

Please note that this file does have the full path of your folders in it, so if you have any sensitive named folders (such as your username or real name) edit those first before submitting. I'm really mostly interested in the folder sizes and number of watched folders

[ValiantBlade]

I haven't been able to recreate this. But I can tell you my folder sizes are titanic, watcher.json

Only the ones on the drives after E were present at the time of incident, I have quite a bit of game storage from old salvaged hard drives. And somehow it doesn't feel like enough.

I may qualify as a data hoarder but that's another story.

[ValiantBlade]

If there was an operation going on, I'd be less concerned, but earlier I wasn't seeing any operations in the watchdogs list.

Anyways, brilliant software and I look forward to seeing where this goes in the future, I'm already recommending this because I've gotten a few hundred GB back, just from first time compression on some drives. This actually makes Compact.exe usable for my scenarios.

[Iridium-IO]

What I'm wondering is if the program ended up storing the list of file names/details of all the folders multiple times without clearing itself in between each analysis loop.

If you hit analyse on your biggest folder while watching the task manager, does the memory usage go up significantly?

I think this is going to be really hard to debug to be honest

[ValiantBlade]

Your theory appears to be on point, because if I just hit analyze manually on the same folders over and over, the memory usage increases, at fixed intervals which are likely based on the size of the folder, apparently without limit.

On an unrelated note, I also seem to have difficulty opening the program from tray sometimes, and the only way I've been able to fix this is deleting the settings.json, just was having that difficulty while testing this.

[ValiantBlade]

I left the analyze on for that specific folder (J:\Games) and my memory usage is slowly climbing to around 900mb. I will edit this with the final size.

Considering it seems to be ballooning far beyond the size I'd observed when I thought it was leaking, maybe it wasn't a leak after all and it was just a REALLY BIG index.

Edit: 1.6GB and counting... It seems to be much slower on mechanical drives because when I did the analyze command on a similarly sized SSD, it took under 3 minutes for 2TB.

Edit edit: that similarly sized folder on the SSD consumed only 250mb during the entire analyze process, now I don't know what's happening. Anyways, I'm gonna do a recount for the mechanical drive folder from a freshly opened instance. In complete fairness, the folder on the mechanical drive is way more complex because it's the drive I shove tiny games on, like Popcap games and stuff.

Edit edit edit: 1GB nearly on the dot for that huge folder. Which means something's not clearing properly between analyzes.

[Iridium-IO]

Thanks for that detective work, I'll have a look over the weekend and try to see where the leak is coming from :)

[dnlck84]

I have quite a bit of game storage [...] And somehow it doesn't feel like enough.

Doesn't that always seem to be the case? *sigh*

[ValiantBlade]

I have quite a bit of game storage [...] And somehow it doesn't feel like enough.

Doesn't that always seem to be the case? sigh

Yep, although TBH I am on the data hoarder side and probably have too much storage.