Sometimes too many bookmarks are removed?

[vaeth]

This bug is opened as a reaction on a review on mozilla.

For some not yet clarified reason, with many duplicate bookmarks and "automatic" selection more than only duplicates have been reported to be removed.

It was conjectured that the number of bookmarks is related, but I tested with huge numbers; therefore, it is unlikely that the number alone is the reason.

I consider it also very unlikely that bookmark dupes instructed firefox to remove a bookmark which was not marked. This leaves the following possible causes for the bug:

1. Too many bookmarks have been marked. I consider it very likely that the code for marking is correct. So very likely this could happen only if none-duplicates had been displayed in one group. Possible reasons.

   • Perhaps "similiar dupes" were selected by accident instead of "exact dupes"?
   • A bug in the code for calculating or displaying the groups of duplicates. So far, I have not been able to detect such a bug. If somebody finds an example please inform me (e.g. by emailing me a bookmark file where the problem occurs).

2. The firefox code for removing a certain bookmark id removes might be buggy and remove the wrong (or more than 1) bookmark (e.g. the bookmark id is not unique as it should be). Possible, but I consider this also quite unlikely.

Edit: For testing 1 (b), it might be useful to use bookmark-dupes-2.4, because this version displays the number of marked bookmarks (you do not have to actually remove bookmarks for counting...).

This version is currently only in the master branch (and you have to load it with about:debugging), because I will wait some days for the spanish and chinese translation before publishing it on mozilla.

[vaeth]

bookmark-dupes-2.4 is now released on mozilla

[yorksille]

Almost 7000 bookmarks and over 100 folders, subfolders, subsub folders, etc. which FF screwed up long ago with tons of dups -- bookmarks AND folders. Finally trying your extension to see if I can clean all this up. Some dupes in different folders are intentional.

Anyway, I tried BD on one folder. It deleted almost everything. Most were not dups.

Here's one example. It deleted both of them along with most of the rest of the bookmarks in this folder. (I restored from backup.)

[yorksille]

[vaeth]

1. Did you enable expert mode? (I wonder how you get the few with the common url...)
2. Are you sure that there are really no two bookmarks with the same name and ip adress in the same folder?
3. Do you see the same list when you press the "Dupes" button again?

I never have seen the exactly same bookmark reported twice in a window when it was not really present twice. If this is really the case (and the answers to the question are N - Y - Y), I have only the explanation that firefox has completely messed up your stored bookmarks and reports them in a loop or something similar. As I am still not able to reproduce the problem, I do not know how to help.

[yorksille]

Hi Martin—

Do you have a paid version of this? If you can fix it this would be worth paying for! Or if not I’ll make a good donation if you can solve it!

1. I enabled, then disabled expert mode (just to see what it looks like).

2. I did a search on the dups page and there are no other bookmarks with that title or url.

3. Yes.

When xmarks when dead years ago I switch to using FF to save my bookmarks, 1000s of them. After enabling sync on all my devices it went crazy, generating loads of duplicate folders and urls. I’m just now trying to see if I can clean it up.

What if I sent you my bookmarks file. Would that help? I would just want to be sure you permanently delete it after testing it.

-- Kroy Ellis 954-946-8539 http://www.KroyTech.com www.KroyTech.com

From: Martin Väth @.> Sent: Monday, October 10, 2022 2:31 PM To: vaeth/bookmarkdupes @.> Cc: yorksille @.>; Comment @.> Subject: Re: [vaeth/bookmarkdupes] Sometimes too many bookmarks are removed? (#22)

1. Did you enable expert mode? (I wonder how you get the few with the common url...)
2. Are you sure that there are really no two bookmarks with the same name and ip adress in the same folder?
3. Do you see the same list when you press the "Dupes" button again?

I never have seen the exactly same bookmark reported twice in a window when it was not really present twice. If this is really the case (and the answers to the question are N - Y - Y), I have only the explanation that firefox has completely messed up your stored bookmarks and reports them in a loop or something similar. As I am still not able to reproduce the problem, I do not know how to help.

— Reply to this email directly, view it on GitHub https://github.com/vaeth/bookmarkdupes/issues/22#issuecomment-1273675619 , or unsubscribe https://github.com/notifications/unsubscribe-auth/A3QYHISXT6DHI75R47VLNWTWCROFBANCNFSM4EHI5CBA . You are receiving this because you commented. https://github.com/notifications/beacon/A3QYHIUEK7WSOU4KY2KDS3LWCROFBA5CNFSM4EHI5CBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOJPVL6YY.gif Message ID: @. @.> >

[vaeth]

After you mention xmarks and how your duplicates arose, it seems now more likely to me that your bookmarks file is seriously hosed (and e.g. doing something inconsistent when reading and writing).

We can try of course, though I will have time for non-trivial experiments only on the weekend, and I am not optimistic that I can find a solution if my above conjecture turns out to be correct. If you are willing to share the file, please send it by pm (briefly before the weekend if you do not want to risk that it is on my disk until then.)

[yorksille]

Ok, maybe it is that the bookmarks file is messed up. It doesn’t sound too likely that you will be able to fix it, so I won’t bother with sending over. Thanks for looking into it! If you get any ideas on how to fix it please let me know.

Thanks!

Kroy

From: Martin Väth @.> Sent: Monday, October 10, 2022 3:04 PM To: vaeth/bookmarkdupes @.> Cc: yorksille @.>; Comment @.> Subject: Re: [vaeth/bookmarkdupes] Sometimes too many bookmarks are removed? (#22)

After you mention xmarks and how your duplicates arose, it seems now more likely to me that your bookmarks file is seriously hosed (and e.g. doing something inconsistent when reading and writing).

We can try of course, though I will have time for non-trivial experiments only on the weekend, and I am not optimistic that I can find a solution if my above conjecture turns out to be correct. If you are willing to share the file, please send it by pm (briefly before the weekend if you do not want to risk that it is on my disk until then.)

— Reply to this email directly, view it on GitHub https://github.com/vaeth/bookmarkdupes/issues/22#issuecomment-1273703543 , or unsubscribe https://github.com/notifications/unsubscribe-auth/A3QYHISU6CLKSJ74OCUMWUDWCRSCDANCNFSM4EHI5CBA . You are receiving this because you commented. https://github.com/notifications/beacon/A3QYHIXE4JHTKNR5DTLSZCTWCRSCDA5CNFSM4EHI5CBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOJPVSY5Y.gif Message ID: @. @.> >

[vaeth]

What I would try is to copy the latest bookmark backup file from the folder. Then start up with a completely clean profile (in particular, not yet switching on syncing) and restore the backup file: If you have good luck, the backup file itself is not broken, and then tools like bookmarkdupes might work (or might not even be necessary anymore). Alternatively, instead of the backup files, you can try to export/import the bookmarks are HTML though that will be more lossy. A problem is: Once you switch on sync, the issue might return as the broken data might be on the server...

[yorksille]

Ok, these are worth trying. Thanks Martin!

From: Martin Väth @.> Sent: Monday, October 10, 2022 3:17 PM To: vaeth/bookmarkdupes @.> Cc: yorksille @.>; Comment @.> Subject: Re: [vaeth/bookmarkdupes] Sometimes too many bookmarks are removed? (#22)

What I would try is to copy the latest bookmark backup file from the folder. Then start up with a completely clean profile (in particular, not yet switching on syncing) and restore the backup file: If you have good luck, the backup file itself is not broken, and then tools like bookmarkdupes might work (or might not even be necessary anymore). Alternatively, instead of the backup files, you can try to export/import the bookmarks are HTML though that will be more lossy. A problem is: Once you switch on sync, the issue might return as the broken data might be on the server...

— Reply to this email directly, view it on GitHub https://github.com/vaeth/bookmarkdupes/issues/22#issuecomment-1273714981 , or unsubscribe https://github.com/notifications/unsubscribe-auth/A3QYHIRGO5I2JLMIH4J2QXDWCRTRPANCNFSM4EHI5CBA . You are receiving this because you commented. https://github.com/notifications/beacon/A3QYHITSEBFLWJH2GVKZQ53WCRTRPA5CNFSM4EHI5CBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOJPVVSJI.gif Message ID: @. @.> >

[vaeth]

I realized that there is also a misunderstanding:

It deleted both of them along with most of the rest of the bookmarks in this folder.

From each group of duplicates all are displayed. If you selected all of them for deletion, of course all of them are deleted, and none will be left. You must leave at least one of them in each group.

[yorksille]

2 (above) was unchecked as with all the others unchecked bookmarks that were deleted. I saw one of each dup (of those I looked at) left unchecked before I deleted what was checked. (I only copied this one example.) As you said, apparently a problem with the FF file.

[yorksille]

I exported the bookmarks to HTML. It's a mess. All kinds of folders and URLs duplicated that aren't showing that way in the bookmarks, old folders and URLs deleted long ago that are there, etc. I'll try importing to Chrome and see what I get and if that doesn't work will try your suggestion. Thanks for the ideas on how to fix this!