Scanning a 2TB of photos hangs after a day or two on Mac OS

[jasontitus]

Bug Description Scanning for near duplicates on a large volume (1.2M photos - ~3TB) of old photos seems to work fine for a day or so but that hangs at 65% of the way after hashing ~800k photos. Attaching STDERR which showed panics and such earlier in the process but nothing bad recently.

Steps to reproduce: Scan large volumes of old photos, some of which are likely corrupt.

Terminal output (optional):

Processing took 805.932ms
Processing took 784.06525ms
Processing took 919.753417ms
Processing took 249.359686958s
Processing took 423.541633291s
Processing took 413.8683325s
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
thread '<unnamed>' panicked at czkawka_core/src/common.rs:271:50:
processing successful: Error { code: -2 }
Processing took 318.612774542s
Processing took 339.120791791s
Processing took 226.41121325s
Processing took 270.202117292s
Processing took 267.398843s
Processing took 295.143028708s
Processing took 319.169452333s

<details>
<summary>Debug log</summary>

# UNCOMMENT DETAILS AND PUT LOGS HERE

</details>

cargo run --release --bin czkawka_gui --features "heif,libraw"

warning: struct Hamming is never constructed --> czkawka_core/src/similar_videos.rs:60:8	60	struct Hamming;	^^^^^^^

= note: #[warn(dead_code)] on by default

warning: czkawka_core (lib) generated 1 warning Compiling czkawka_gui v7.0.0 (/Users/jasontitus/experiments/czkawka/czkawka_gui) Finished release profile [optimized] target(s) in 6.28s Running target/release/czkawka_gui 16:06:36.895 [INFO] czkawka_core::common: App version: 7.0.0, release mode, rust 1.78.0 (2024-04-29), os Mac OS 14.4.1 [aarch64 64-bit], 20 cpu/threads 16:06:37.283 [INFO] czkawka_gui: Set thread number to 20

System

• Czkawka/Krokiet version: czkawka_core::common: App version: 7.0.0, release mode, rust 1.78.0 (2024-04-29), os Mac OS 14.4.1
• OS version: Mac OS 14.4.1
• Installation method: build from pull a few days ago

[jasontitus]

[jasontitus]

With full backtraces - thread '' panicked at czkawka_core/src/common.rs:568:51: called Result::unwrap() on an Err value: SystemTimeError(40.598ms) stack backtrace: 0: 0x102a4c9a4 - ::fmt::he735dbc19bf16515 1: 0x102a6f3f4 - core::fmt::write::hef5cddb56c038e23 2: 0x102a384c8 - std::io::Write::write_fmt::h0dc631791748dd86 3: 0x102a4c7fc - std::sys_common::backtrace::print::h104bc4229e84cb49 4: 0x102a60ec8 - std::panicking::default_hook::{{closure}}::he5ba2bc230f0bb9e 5: 0x102a60c04 - std::panicking::default_hook::hf55ea1ed61f9a330 6: 0x102a61264 - std::panicking::rust_panic_with_hook::h29cd1affa2bec942 7: 0x102a4ce84 - std::panicking::begin_panic_handler::{{closure}}::h20e35afb75c4d3ea 8: 0x102a4cbc4 - std::sys_common::backtrace::rust_end_short_backtrace::h3caf01002b4ebcc2 9: 0x102a60fac - _rust_begin_unwind 10: 0x102abcfe0 - core::panicking::panic_fmt::hd47b186e02bbaa3c 11: 0x102abd354 - core::result::unwrap_failed::hf18ffba820174349 12: 0x1024c6444 - std::sys_common::backtrace::__rust_begin_short_backtrace::h1a3f77febd6da073 13: 0x1025664b4 - core::ops::function::FnOnce::call_once{{vtable.shim}}::hf7f08a4403702c18 14: 0x102a4708c - std::sys::pal::unix::thread::Thread::new::thread_start::he47ee5f2ef45b057 15: 0x18cccaf94 - pthread_joiner_wake