search

jbinkley60 / plugin.program.kscleaner

GNU General Public License v2.0
9 stars 2 forks source link

Add a check for duplicates (over smb://) #6

Open rafi-d opened 1 month ago

rafi-d commented 1 month ago

Hi, I have a few smb driver defined, and I cannot find a way to get rid of duplicates with capitalization issue. Example: image

Can you please add a check to analyze and remove one of those too? ATM, the cleaner does not work for them... :(

Thanks!

PS: Probably got added in the past when I defined two resources for the same path...

jbinkley60 commented 1 month ago

I'll see what I can do yo make the queries case insensitive.

jbinkley60 commented 1 month ago

The duplicate analyzer is properly finding the duplicate records, regardless of case, but currently deletion of duplicates is not part of the cleaner. Are you saying that these truly are duplicates with different paths ?

The challenge of automated removal would be knowing which duplicate to remove. I'll add this to the forum for discussion regarding options for this functionality.

rafi-d commented 1 month ago

The duplicate analyzer is properly finding the duplicate records, regardless of case, but currently deletion of duplicates is not part of the cleaner. Are you saying that these truly are duplicates with different paths ?

Unfortunately, yes :( This is how it looks here (on Windows): image

I've opened an issue on Kodi github, but it was never dealt with :( https://github.com/xbmc/xbmc/issues/25053

Also, I believe there were threads about that in the forums, like https://forum.kodi.tv/showthread.php?tid=365528&pid=3073732#pid3073732

Regarding deletion, I think it will be useful. Maybe the way to go is to compare the path name(s) to the source path(s) and delete those duplicates that do not match any source path. If they both match sources (there is a duplicate source) - might be a good idea to warn, and ask if to delete it too...

rafi-d commented 1 month ago

I do understand that you can get use-cases were duplicates are due to some multiple mapped path (in Windows) , but he most common use case is either different letters' case, and a path being both local drive and smb: . I believe both can be easily detected, especially the first one...

jbinkley60 commented 1 month ago

Based upon what you are seeing and saying, the detection appears to be working properly. The ask then is to add a cleaning function to the duplicate detection. Would the scenario be that you want to delete a duplicate path and everything in it or selectively delete certain items from a duplicate path ?

rafi-d commented 1 month ago

I think - delete the path and all the duplicated entries in that duplicated path. I assume that path will be the one that has no exact source defined for it.

BTW, the only way I found to do that in Kodi, is: 1 remove the source (used for one of the dups) completely, and tell Kodi to remove everything in it

  1. Still, only one of the dups is removed.
  2. Now, clean the library. Now everything is really gone
  3. Now, create the source again, define the type, and re-scan everything

So, quite long and complicated....

rafi-d commented 1 month ago

There is one more use case, which I've encountered: When you define the content as "path names are the movie names" - and you have additional random files in it , like xxx.dat or sample.mkv, Kodi lists the other files as library videos as well with the same movie name. Even after you delete the entries it re-scrape/scans them. I think this use case is difficult to auto-delete , but you can still - show those in the detection log. I am still not sure how to set up advanced setting to NOT show those:

https://forum.kodi.tv/showthread.php?tid=379046