search

jimmejardine / qiqqa-open-source

The open-sourced version of the award-winning Qiqqa research management tool for Windows
GNU General Public License v3.0
373 stars 61 forks source link

long running FolderWatcher add session + recover PDFs from library: 100% CPU and random application crash (error 0xc0000374) #98

Closed GerHobbelt closed 4 years ago

GerHobbelt commented 4 years ago

Sometimes it takes a few seconds, sometimes ages before it happens: you ALMOST always end up with something like this in the log (v82pre):

20191006.105355 [Q] INFO  [Daemon.Maintainable:BackgroundWorkerDaemon.DoMaintenance_Infrequent] [172.801M] Indexing document 1FC78F204EF19FF95BC260B9BC9A9B57CCE8E
20191006.105355 [Q] DEBUG [Daemon.Maintainable:BackgroundWorkerDaemon.DoMaintenance_Infrequent] [172.916M] Using calculated PDF page count for file D:\Qiqqa\base\INTRANET_EF52564A-831D-42F2-B956-815CF0418C08\documents\1\1FC78F204EF19FF95BC260B9BC9A9B57CCE8E.pdf
20191006.105356 [Q] INFO  [Daemon.Maintainable:BackgroundWorkerDaemon.DoMaintenance_Infrequent] [174.430M] Indexing document 1FC11E81DE5201255F4DB72F18EF011931DB595
20191006.105356 [Q] DEBUG [Daemon.Maintainable:BackgroundWorkerDaemon.DoMaintenance_Infrequent] [174.537M] Using calculated PDF page count for file D:\Qiqqa\base\INTRANET_EF52564A-831D-42F2-B956-815CF0418C08\documents\1\1FC11E81DE5201255F4DB72F18EF011931DB595.pdf
20191006.105356 [Q] INFO  [Daemon.Maintainable:BackgroundWorkerDaemon.DoMaintenance_Infrequent] [172.161M] Indexing document 1FBD5921AC058917E69BD6D94A64E4FCD19E2DA
20191006.105356 [Q] DEBUG [Daemon.Maintainable:BackgroundWorkerDaemon.DoMaintenance_Infrequent] [172.267M] Using calculated PDF page count for file D:\Qiqqa\base\INTRANET_EF52564A-831D-42F2-B956-815CF0418C08\documents\1\1FBD5921AC058917E69BD6D94A64E4FCD19E2DA.pdf
The program '[22320] Qiqqa.exe: Program Trace' has exited with code 0 (0x0).
The program '[22320] Qiqqa.exe' has exited with code -1073740940 (0xc0000374).

Adding logging to CountPDFPages() API seems to change the behaviour, but the crashes don't go away, they just occur at differen times. Besides, the moment of crash is highly unpredictable anyway, which is indicative of something seriously nasty and deep-rooted, e.g. timing/locking issues with multithreaded object/variable use (see #96 for one example of that)

Analysis so far / Workaround = fix? 🤔🤔🤔 ❓

After running many of the sessions, which all abort Qiqqa suddenly and without any noise, just poof! and Qiqqa's gone!, every time with the same error 0xc0000374 which the InterWebz identify as memory heap corrupted, it smells like Sorax lib usage inside CountPDFPages() is a memory leaker (???) and corruptor as often the amount of (logged) memory-used-by-the-app slowly grows just before the final log line says '0xc0000374'.

Disabling the Sorax invocation in that API apparently removes the problem of sudden death, as now LuceneNet starts to throw hairballs after intense runs of several hours -- which never succeeded before due to the error code 0xc0000374: longest run was about 30 mins then.

GerHobbelt commented 4 years ago

Closing and decluttering the issue list so it stays workable for me: fixed in https://github.com/GerHobbelt/qiqqa-open-source mainline=master branch, i.e. v82pre3 experimental release and beyond.