Clementine creates duplicates in library

[Clementine-Issue-Importer]

From apgrant2 on April 19, 2011 03:33:37

What steps will reproduce the problem? 1.Using Clementine over time 2.Updating Music library automatically 3.Perhaps it is a copying songs to device issue? What is the expected output? What do you see instead? It is expected to have only one song appear in Clementine when there is only one such song on my hard disk however Clementine seems to be creating multiple copies of songs. I haven't been able to narrow down the issue to any one event. It just seems that over time a duplicate is added. This goes for every album in my library. Some have 5 dupes. Others have 10 dupes. Though none have more than 10. What version of the product are you using? On what operating system? I'm using Clementine 0.7 r3146 and QT 4.70. This being used on Ubuntu 10.10. Please provide any additional information below. The dupes began at two and have grown larger over time until they've reached 10 dupes per song on some albums. When I use another music player such as Banshee or Rhythmbox there are no dupes. Also, when I check my Music folder for dupes there are no dupes. I've been searching the forums and such for any leads but haven't found any. Also, I've checked this site for someone reporting the same kind of issue and have also come up empty. If I makes duplicatesam mistaken in any way please advise. I want nothing more than to continue to use Clementine.

Attachment: Selection_005.png

Original issue: http://code.google.com/p/clementine-player/issues/detail?id=1785

[Clementine-Issue-Importer]

From keirangtp on April 19, 2011 05:17:13

As a workaround try using "Tools / Rescan whole library" - this will probably remove the duplicates but the process may take some time especially if you have a huge library.

Restarting Clementine doesn't help - the duplicates are still there?

Summary: Clementine creates duplicates in library

[Clementine-Issue-Importer]

From apgrant2 on April 19, 2011 10:46:08

I've tried that in the past and it hasn't worked but I'm hoping it does this time and unfortunately restarting it doesn't help either.

[Clementine-Issue-Importer]

From keirangtp on April 20, 2011 00:58:40

When did you try it? 0.7 has a whole library rescan feature, 0.6 had only "refresh changed folders" feature and that one wouldn't fix it. http://code.google.com/p/clementine-player/wiki/LibraryScanning

[Clementine-Issue-Importer]

From apgrant2 on April 20, 2011 09:39:56

I tried it recently with version 0.7's library rescan feature and no joy. I've also restarted Clementine a bunch and have been regularly updating the version from my update manager as they arise but alas I still have dupes all over the place. It's very strange.

[Clementine-Issue-Importer]

From chuck.smith08 on April 20, 2011 13:00:48

Just saw this issue.....

I reported as issue 1790 and am having the same issue, Re-scan does not work....

Probably should merge these issues....

[Clementine-Issue-Importer]

From davidsansome on April 20, 2011 13:17:02

Issue 1790 has been merged into this issue.

[Clementine-Issue-Importer]

From chuck.smith08 on April 20, 2011 17:50:06

I tracked down a problem that I believe is where the duplicates are coming from. My music folder is:

/media/storage/Music

Inside that folder there is a link to /media/storage/Music, etc. The resulting directory structure is:

/media/storage/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music/Music

I can only assume clementine created but I do not know how or when..

[Clementine-Issue-Importer]

From apgrant2 on April 22, 2011 02:50:09

Hmm.. that does sound plausible considering that when I choose the option to delete the dupes from disc in Clementine it drops an error regarding being able to delete the songs. I assume it's because the songs aren't actually there but are rather somehow linked to the original song. Also however, how or when or why I have no idea.

[Clementine-Issue-Importer]

From apgrant2 on April 22, 2011 03:02:26

A rather inelegant solution depending on how large your collection is to choose the "Organize Files" option in Clementine with "Overwrite existing files" clicked and "Delete the original Files" selected in the "After Copying" section located at the top of the menu. Take note however that I have not tested for unforeseen after effects. So, use at your own risk.

[Clementine-Issue-Importer]

From schizosfera on June 03, 2011 07:26:26

i had a similar problem when duplicates appeared in my library. usually one of the duplicate entries would have an invalid path. that was because of some reason, clementine stored escaped file url's in the database (spaces were escaped as %20). doing library rescans didn't remove any of the invalid entries (which displayed as duplicates). manually deleting the entries from the db fixed my problem.

[Clementine-Issue-Importer]

From schizosfera on June 04, 2011 03:24:26

i just noticed that (mp3, cue) pairs create duplicates in my library (probably when rescanning the library). i'm not sure if it's actually the same issue, but it definitely looks like the same symptom.

Attachment: mp3 with cue duplicates.png

[Clementine-Issue-Importer]

From adam.chrzascik on June 06, 2011 05:36:45

To reproduce the bug(?) on Win7 x64:

Install r3341 version over (not clean install) existing stable 0.7.1 release with library in place. Now rescan the library. For every song you will have two entries in library view, with paths in properties shown differently :

D:\Music\U2\How To Dismantle An Atomic Bomb\01 - Vertigo.mp3 \d\Music\U2\How To Dismantle An Atomic Bomb\01 - Vertigo.mp3

Restart computer. Now paths look like this:

file:\D:\Music\U2\How%20To%20Dismantle%20An%20Atomic%20Bomb\01%20-%20Vertigo.mp3 file:\D:\Music\U2\How To Dismantle An Atomic Bomb\01 - Vertigo.mp3

Now run stable installer again, rescan the library, and you will get third entry with correct path:

D:\Music\U2\How To Dismantle An Atomic Bomb\01 - Vertigo.mp3 file:\D:\Music\U2\How%20To%20Dismantle%20An%20Atomic%20Bomb\01%20-%20Vertigo.mp3 file:\D:\Music\U2\How To Dismantle An Atomic Bomb\01 - Vertigo.mp3

Cheers,

[Clementine-Issue-Importer]

From schizosfera on June 06, 2011 05:51:08

it seems like the escaped file URIs (those containing the %20's )are the new standard entries. a migrate for the other "old" DB is not available yet, but is planned (so i heard) - probably with the next release.

@adam.chrzascik: the entries starting with file:/// are probably the working/correct ones. strange thing that on windows they use backslashes instead of slashes in URIs...

[Clementine-Issue-Importer]

From keirangtp on June 11, 2011 12:39:13

Labels: Milestone-1.0

[Clementine-Issue-Importer]

From ColdAssassin008 on June 15, 2011 00:44:13

Im also having this problem. It seems that somehow it is getting the wrong directory path.

C:\Users\ColdAssassin\Music\Jap\M-Flo\m-flo - inside-WORKS BESTII-\disk1\04 I Know.mp3 This is my real path C:\Users\ColdAssassin\Music\Jap\m-flo\m-flo - inside-WORKS BESTII-\disk1\04 I Know.mp3 and this is the duplicate path name. Oh I figured out what the problem was. somehow the dash was causing the duplicates so i just remove "M-Flo" and made it "MFlo" and the duplicates were removed.

[Clementine-Issue-Importer]

From nalle.karhulitos on August 03, 2011 10:56:17

I got rid of my duplicates by $ cd .config/Clementine $ cp clementine.db clementine.db.bk $ sqlite3 clementine.db sqlite> delete from songs where filename like 'file:///%'; sqlite> .quit

[Clementine-Issue-Importer]

From graeme@thelittlehippocompany.co.uk on August 04, 2011 13:14:11

I had an issue earlier that I think belongs to this ticket: I used the organise files function to "move" an album (after fixing the artist tag) from "Vnv Nation" to "VNV Nation". And while my OS is case-sensitive (OSX), I guess sqlite might not be and so I had duplicate records for each track in the album (both the old and new artist name).

A full library scan would probably have fixed it but instead I "cheated" and renamed the artist folder to something else (not using clementine), waited for auto-update to remove the songs and then changed it back so after the second auto-update it only found the one copy of each track.

[Clementine-Issue-Importer]

From keirangtp on August 18, 2011 11:20:22

Issue 2086 has been merged into this issue.

[Clementine-Issue-Importer]

From davidsansome on September 24, 2011 04:24:06

Issue 2248 has been merged into this issue.

[Clementine-Issue-Importer]

From davidsansome on November 12, 2011 18:01:53

Labels: -Milestone-1.0

[Clementine-Issue-Importer]

From john.maguire on January 05, 2012 03:24:34

Issue 2566 has been merged into this issue.

[Clementine-Issue-Importer]

From spackmat on February 06, 2012 03:42:27

Having the same trouble on all my 3 installations: Seems as if neraly every startup of Clementine rescans the library and adds another duplicate entry for every track. In all cases the music is stored on a network share and the library base is given as a UNC-path (\homeserver\music), the computers are running Win7 x86 and x64. The rescan takes much time just like a complete new scan.

I rebuilt the whole library several times, but after promising restarts with a quick (a few seconds) library scan without creating duplicates it always fell back into the behaviour described above rendering the library-feature completely useless.

[Clementine-Issue-Importer]

From jen.mclean8 on March 14, 2012 04:00:47

Exactly the same issue. Library on network drive, duplicates on restart. Remove and re-add library, fine until next restart.

[Clementine-Issue-Importer]

From ajgallego on March 24, 2012 09:50:23

For me the solution was the one at the comment #16

[Clementine-Issue-Importer]

From ibefraggle on May 08, 2012 10:30:33

Confirming #22/23. Win7 x64, some music on network share, some on local hard drive. Anytime the library updates (folder contents change, manually from Tools menu, on startup), new duplicate library entries of every song in the network share are created. These library entries have no apparent way of being removed, short of removing the network share from the library folder list in Preferences. Deleting the files does not help.

Wondering if this is a seperate issue to 1785.

[Clementine-Issue-Importer]

From sebastiangoncalves on July 13, 2012 10:42:31

Got the same issue after upgrading from 10.10 to 11.04 (yesterday).

I solved in a simple way: Tools -> Preferences -> Music Library, then select folder with mouse, press and finally selecting the original folder

[Clementine-Issue-Importer]

From ayatzid on December 11, 2012 10:34:28

and my saved playlists...gone with the wind...

[Clementine-Issue-Importer]

From ivan.gojkovic on December 13, 2012 13:42:14

Sorry, people. This issue is too long with no reply. My clementine is going to trash for good. RhitmBox seams to be best for linux, and Aimp for Win. Ciao...

[Clementine-Issue-Importer]

From spackmat on December 13, 2012 14:28:29

Same here, I love the featureset of Clementine, but with music on a windows network share it is really unstable, not just with the persisting problem of the many duplicate entries in the library. It starts relly slow and hangs constantly for several seconds with no response. Sad, but I have to look after a new player again.

[Clementine-Issue-Importer]

From dtverder on January 02, 2013 23:50:55

I have started using Clementine 1.1 on Windows 7 (previously I mainly used it on Fedora) and I am also starting to get lots of duplicates on large playlists, e.g. 'all tracks' or 'all unplayed'. The tracks it happens to seem to be the ones not properly tagged and with basic file names such 'Track01.mp3'. The duplicates seem to occur in multiples of five, usually it duplicates the track ten times but sometimes it is five or fifteen.

[Clementine-Issue-Importer]

From tommy.carstensen on February 23, 2013 08:17:55

I am having the same problem on Linux Mint with Clementine 1.1. I don't like Rhythmbox, Spotify, Amarok and iTunes, so sadly I have to revert to Windows Media Player, until this bug is fixed. I tried the fix suggested in #16, but that didn't work.

[Clementine-Issue-Importer]

From g.legoc on April 08, 2013 06:05:08

Same problem after some weeks using Clementine 1.1 on Win7x64. It seems to happen when Clementine is open while I copy, convert, cut music to an USB device.

[Clementine-Issue-Importer]

From davidsansome on April 10, 2013 04:48:35

Issue 3605 has been merged into this issue.

[Clementine-Issue-Importer]

From Ninjawailer on May 16, 2013 08:42:03

I'm getting this issue too for what it's worth. I've got duplicates for each rescan.

[Clementine-Issue-Importer]

From s.v.valor on May 30, 2013 19:53:28

The same is happening to me on pclinuxos - the song list shows duplicates even though there are no duplicates on the drive. It is a bit of a nuisance but when the layers of duplicates get to the point of being really irritating, I go into synaptic and remove clementine, remove the Clementine directory in .config, then reinstall and rescan. That fixes things for a while.

It's just a little effort, and Clementine is worth it - great little music player!

[Clementine-Issue-Importer]

From jan.ekbom on June 30, 2013 02:02:16

I get duplicates in Clementine version 1.1.1 on Linux Mint 15. This is a little annoying.

[Clementine-Issue-Importer]

From breakpointmobile on July 12, 2013 17:15:42

OK, I feel pretty sure I know what's going on with this: certain actions are starting TWO RESCANS, in SEPARATE THREADS, at the same time.

Given a FULLY MULTI-CORE CPU (such as a Core i7) and a Windows 8 box with an external hard drive and a single (exactly one) folder "E:\music\FLAC" (with subhierachy Artist\Album\Track Name.flac), take the following steps to reproduce with minimum action only (perform NO ACTION not specified explicitly in this list):

1. Kill all instances of Clementine, including background processes.
2. Unplug USB hard drive, wait 5 seconds, replug USB hard drive, wait 10 seconds.
3. Start Clementine.
4. Go to preferences and remove ALL (ALL [ALL]) paths from the Library.
5. DO NOT CLICK OK TO CLOSE THE PREFERENCES PANE.
6. WATCH THE STATUS BAR. INTENTLY. For thirty seconds or so.
7. You will eventually observe "Backing up the database." Note this, as testing shows it is likely relevant. That is a slightly separate issue but it is important. NOTE WHETHER OR NOT YOU SEE THIS.
8. CLEAR "Update the library when Clementine starts".
9. CLEAR "Monitor the library for changes."
10. Click OK.
11. Verify that the Library pane shows "Your library is empty!"
12. In the Music menu, select Quit.
13. Verify that Clementine has fully quit using Task Manager or Process Explorer.
14. Restart Clementine.
15. Verify that the Library is still empty.
16. Start Clementine.
17. Open Preferences.
18. Add EXACTLY ONE AND ONLY ONE path to the Library (in this example, "E:\music\FLAC").
19. DO NOT CLICK OK TO CLOSE THE PREFERENCES PANE.
20. WATCH THE STATUS BAR for 30 seconds or so and note whether or not you see "Backing up the database".
21. KEEP CLEAR "Update the library when Clementine starts".
22. KEEP CLEAR "Monitor the library for changes."
23. Click OK.
24. Watch the status bar CLOSELY.
25. NOTE WHETHER YOU OBSERVE IT GOING 95%, 96%, 95%, 97%, 96%, i.e., flickering back and forth.

If you observe that, you are almost certainly observing two parallel library rebuilds, and I strongly suspect it is this condition that is causing library duplication issues.

The issue should be resolved by proper use of synchronization objects such as mutexes, critical sections, or semaphores.

Regards, Matt Heck

[Clementine-Issue-Importer]

From davidsansome on July 12, 2013 21:06:42

What makes you think any of this is relevant? The progress goes backwards sometimes when it discovers new subdirectories and updates its idea of how much work there is left to do. There is only one library updater thread.

[Clementine-Issue-Importer]

From T.Blumenbach on September 05, 2013 10:21:46

For those using Clementine 1.1.1 on Windows with their music on a network share: Looks like adding the network share as a "real" network drive with a drive letter in Explorer (i. e. "Map Network Drive", select drive letter, enter UNC path) and using that drive letter as your library folder in Clementine prevents duplicates when rescanning the library.

[Clementine-Issue-Importer]

From mccallum2182 on October 28, 2013 21:58:27

I tried the above solution but every time you do a full library re-scan it still produces duplicates.

Any updates on a fix for this issue?

[Clementine-Issue-Importer]

From hilary.bh.wright@wrightfamily.net on November 18, 2013 08:56:18

Hi, I'm having the same issue on a Fedora 19 Linux box running Clementine 1.2 rc4 where the files are stored on a network drive mapped in fstab to the file system containing music organised by artist, album, track but its not the whole library that is duplicated, just some of the subdirectories.

I see that 1.2 final is now released; I'll install that and rebuild the library to see if problem resurfaces.

[Clementine-Issue-Importer]

From hilary.bh.wright@wrightfamily.net on November 18, 2013 12:13:25

Well, I updated to 1.2 final and rebuilt the library. Problem still exists.

[Clementine-Issue-Importer]

From hilary.bh.wright@wrightfamily.net on November 18, 2013 12:30:43

OK, panic over. ONe of the children had created a duplicated directory with some of his mmusic in it. Hence duplicates for some items and not others. So problem resolved.

[Clementine-Issue-Importer]

From Steve.Smiffy88 on November 23, 2013 00:12:42

Hello All. For info: Using Clementine 1.2 on Win7 x64. As soon as I pointed my library to my NAS, addressed as \NAS... , the duplication issues started; previously, I had the library pointed to a drive on the local PC (D:...), without issues. I then mapped the NAS folder as a network drive (Z:...) and pointed the library to that drive, and the duplication issues ceased.

[iangreenleaf]

I've been having duplicates crop up for years. In my case, I think it's related to when I retag and/or rename files. I looked at the database to see what's going on, and there are indeed two entries for the same file location. It seems like the primary difference between the rows might be the performer column. Here's an example:

sqlite> select * from songs where title = 'Winter Song';
title|album|artist|albumartist|composer|track|disc|bpm|year|genre|comment|compilation|length|bitrate|samplerate|directory|filename|mtime|ctime|filesize|sampler|art_automatic|art_manual|filetype|playcount|lastplayed|rating|forced_compilation_on|forced_compilation_off|effective_compilation|skipcount|score|beginning|cue_path|unavailable|effective_albumartist|etag|performer|grouping
Winter Song|The Head and the Heart|The Head and the Heart|The Head and the Heart||8|1|-1.0|2011|Folk||0|162000000000|212|44100|4|file:///mnt/music/Winter%20Song.mp3|1333923299|1333923299|4313708|0|/mnt/storage/Aural/stolen music/tagged/The Head and the Heart/Disc 1/front.jpg||5|29|1427743967|-1|0|0|0|0|98.1034482758621|0||0|The Head and the Heart||The Head and the Heart|
Winter Song|The Head and the Heart|The Head and the Heart|The Head and the Heart||8|1|-1.0|2011|Folk||0|162000000000|212|44100|4|file:///mnt/music/Winter%20Song.mp3|1333923299|1333923299|4313708|0|/mnt/storage/Aural/stolen music/tagged/The Head and the Heart/Disc 1/front.jpg||5|0|-1|-1|0|0|0|0|0|0||0|The Head and the Heart|||

I've got a DB with a ton of duplicates, so I can provide more info if needed for debugging.

[mxa]

I have this issue in Version 1.2.3-1680-g931b6aa.

[davidoskky]

I had this problem and solved it quite easily... the problem consisted in having by error registered twice the same directory in the library. Therefore all the subdirectories were scanned twice. Removing the second entry solved my problem. It would be nice if you made it impossible for this kind of things to happen.

[sorpaas]

Just a side note: if you double-click one of the smart "playlists", the default behavior is to append to current playlist, so it will create duplicates in the current playlist, but not in the library.

[mxa]

Hope this screenshots makes it clearer. When I go to "Show in File Browser" for each of the double entries, it opens the same folder, where there are no duplicates. This affects maybe 5% of the library. Most folders look OK.

[crissimaster]

I can also confirm using 1.3.1-268-g789c492 that drag n drop a file from file browser adds the file(s) twice (double entries).

Also a strange thing: if I select "Remove duplicates from playlist" not only the duplicates get removed, some more ;(