jimmejardine / qiqqa-open-source

The open-sourced version of the award-winning Qiqqa research management tool for Windows
GNU General Public License v3.0
373 stars 61 forks source link

Qiqqa sometimes reports Unexpected exception (OutOfMemory) when using Sniffer with large libraries #19

Closed GerHobbelt closed 5 years ago

GerHobbelt commented 5 years ago

Exception thrown: Insufficient memory to continue the execution of the program

Running Qiqqa in the MSVC debugger and having a quick look at the memory footprint (snapshot inspection), it turns out that in this case the LDAStuff.DocProbability and LDAStuff.TopicProbability both are huge: Count > 3.5 million, Size > 70MB each.

The next memory hog (?) seems to be a WPF component: WpfTap.WpfVisualTreeService --> Dictionary component @ Count ~ 24K and Size ~ 35MB :: I bet that's the visual list of the entire library.

However, the LDAStuff is odd to me as I haven't used Qiqqa Expedition or anything else, except the Sniffer during this session.

2019-08-02-(1)A

GerHobbelt commented 5 years ago

https://stackoverflow.com/questions/14186256/net-out-of-memory-exception-used-1-3gb-but-have-16gb-installed

GerHobbelt commented 5 years ago

Seems fixed as per #33.

This has been an elusive bugger and a lot of time has been spent on IDisposables tracking and other post-partum cleanup matters in the code. memory leaks in C# have always been a horror to track down so I cannot guarantee that it's been fixed. 😞

This has also triggered me to run a Code Analysis Report on the codebase (MSVS: pick Solution > Analyze and Code Cleanup > Run Code Analysis on Solution (Alt+F11) which has led to quite a bit of code inspection and activity - see the CAxxxx commits in the tree.

Considered done; any future work on this should be filed in another issue.

GerHobbelt commented 5 years ago

Closing and decluttering the issue list so it stays workable for me: fixed in https://github.com/GerHobbelt/qiqqa-open-source mainline=master branch, pending #15 / any maintainer rights/actions.