arsenetar / dupeguru

Find duplicate files
https://dupeguru.voltaicideas.net
GNU General Public License v3.0
5.23k stars 412 forks source link

Memory Error and subsequent crash #625

Open eltron247 opened 4 years ago

eltron247 commented 4 years ago

While eliminating dupes, dupeguru crashes. The operation is being conducted over LAN to an SMB share. The login is with saved credentials and thus a signout is not a likely cause. (I was having issues a week ago that this was the cause of.)

Error occurred while attempting to move dupe photos to the recycle bin. (no direct move, no change to linking) Files being trashed are jpg's. Operation was being performed over 2 folders in the same sub directory with the "source" marked as reference and containing awr files. There are a total of 21032 duplicates totaling ~104GB

The system actually hung the graphics (everything went black on all monitors) as well so maybe a GPU / memory conflict? I had to unplug all monitors to get it to reset. The reason this could be is that Adobe Light Room is also running with the open catalog referencing some of the jpg's. No action was being taken on them at the time however. Chrome crashed and kept a black window even after resetting the monitors.

The error report output:

Application Name: dupeGuru Version: 4.0.4 RC

Traceback (most recent call last): File "hscommon\gui\progress_window.py", line 101, in pulse File "core\app.py", line 323, in _job_error File "hscommon\jobprogress\performer.py", line 43, in _async_run File "core\app.py", line 223, in _do_delete File "core\results.py", line 298, in perform_on_marked File "core\app.py", line 220, in op File "core\app.py", line 242, in _do_delete_dupe File "core\app.py", line 411, in clean_empty_dirs File "hscommon\path.py", line 236, in wrapper File "hscommon\path.py", line 226, in wrapped File "hscommon\util.py", line 340, in delete_if_empty File "hscommon\path.py", line 175, in listdir File "hscommon\path.py", line 175, in MemoryError

OS in use: Windows 10 Enterprise x64

Source: Not built from source, using the provided binary.

Hope this helps, if you need anything else please let me know.

eltron247 commented 4 years ago

Just checked again and Light Room actually crashed out completely. Adobe Lightroom Classic CC 8.2.1 [1206193] Camera Raw 11.2.1 Windows 10 Enterprise 10.0.17134 16GB RAM

Edit: added LR, OS, and RAM info

rogerclarkmelbourne commented 4 years ago

@eltron247

Did Dupeguru or another application crash.

If another application crashed, then its not an issue with Dupeguru and you should close this issue.

eltron247 commented 4 years ago

Yes, dupeguru crashed as well. The crash dump is listed at the top.

moatmai commented 4 years ago

Application Name: dupeGuru Version: 4.0.3

Traceback (most recent call last): File "C:\Users\arsen\github\dupeguru\hscommon\gui\progress_window.py", line 101, in pulse File "C:\Users\arsen\github\dupeguru\core\app.py", line 323, in _job_error File "C:\Users\arsen\github\dupeguru\hscommon\jobprogress\performer.py", line 43, in _async_run File "C:\Users\arsen\github\dupeguru\core\app.py", line 780, in do File "C:\Users\arsen\github\dupeguru\core\scanner.py", line 137, in get_dupe_groups File "C:\Users\arsen\github\dupeguru\core\scanner.py", line 82, in _getmatches File "C:\Users\arsen\github\dupeguru\core\engine.py", line 269, in getmatches_by_contents MemoryError

This happened while comparing two folders with large files (4GB).