search

eyalroz / removedupes

Remove Duplicate Messages
https://addons.thunderbird.net/en-US/thunderbird/addon/removedupes/
Other
80 stars 6 forks source link

Crashing Thunderbird with large collection of E-Mails #219

Open Jurgster opened 2 weeks ago

Jurgster commented 2 weeks ago

Hi, So here is the config: Thunderbird 115.11.1 (64-bit) Remove Duplicate Messages Version 0.5.4b5

Apart from the fact that the scan requires over 24 hrs, once it has sorted and I say it can delete it does not take to long to crash Thunderbird. I will mention we are talking about a large collection of E-Mails, I am over 25k. Maybe a status file would help with a pickup from crash situation to cut-out repeated start from the beginning?

Respect to the addon still.

eyalroz commented 2 weeks ago

That is quite weird. Many users typically scan 100 K messages and much more - within a minute. Is it possible that you are comparing bodies, and use very few other comparison criteria? Otherwise, there is something wrong going on with TB's access of the mail folders.

Jurgster commented 2 weeks ago

Hi, General Settings: Skip IMAP messages marked 'Deleted' Search subfolders of selected top folder(s) Review search results before deletion Default action for duplicate messages: Delete permanently

Message Comparison settings: Author('From') Recipients ('To') CC List Message ID Send time Subject

eyalroz commented 2 weeks ago

Weird... are your messages on an IMAP server? If so, can you try copying (all) of your messages to a subfulder of some (new) local folder, and trying the duplicate removal on that folder?

Jurgster commented 1 week ago

Hi, Had a lot to do, so got back to trying to solve the hiccups. Found out a probable reason why things are not working. First disabled Review search results before deletion things went considerably faster but it is still crashing most likely due to bad hashing / sorting / etc... Screenshots will hopefully give the gist. Before disabling Review search results before deletion: thunderbird hash mesages

thunderbird hash mesages ergibnis After: thunderbird hash mesages ergibnis folder However the truth is the folder has: thunderbird hash mesages ergibnis folder wirklich

Trying to repair bought nothing.

eyalroz commented 1 week ago
  1. Can you verify the folder and subfolders together only contain 95 messages?
  2. Is that an IMAP folder?
  3. If so, what happens when you copy that folder, with subfolders, into a local folder, and run a dupe search there?

Finally, I'll say that when you're seeing potentially-buggy behavior, I would recommend moving to trash or another folder rather than deleting permanently - to be on the safe side.

Jurgster commented 1 week ago

Local Folders Inbox/subfolder level1/subfolder level2 Inbox after filter empty with 15 subfolder leve1 subfolder level1 9 Messages and 10 subfolder level2 1)8 Messages 2)122 Messages 2 subfolders 3)given subfolder level2 with 95 Messages has 95 Messages and no further subfolder 4)4 Messages 3 subfolders 5)125 Messages 6)54 Messages 7)2480 Messages 8)44 Messages 9)122 Messages 10)1722 Messages 2 subfolders I will stop at that there are many other subfolders.

As one can see no to IMAP