Plex DupeFinder is a python script that finds duplicate versions of media (TV episodes and movies) in your Plex Library and tells Plex to remove the lowest rated files/versions (based on user-specified scoring) to leave behind a single file/version.
Duplicates can be either in bulk (automatic) or on-by-one (interactively).
Click to enlarge.
Python 3.6+.
Required Python modules (see below).
Note: Steps below are for Debian-based distros (other operating systems will require tweaking to the steps).
Install Python 3 and PIP
sudo apt install python3 python3-pip
Clone the Plex DupeFinder repo.
sudo git clone https://github.com/l3uddz/plex_dupefinder /opt/plex_dupefinder
Find your user & group.
id
Fix permissions of the Plex DupeFinder folder (replace user
/group
with yours).
sudo chown -R user:group /opt/plex_dupefinder
Go into the Plex DupeFinder folder.
cd /opt/plex_dupefinder
Install the required python modules.
sudo python3 -m pip install -r requirements.txt
Create a shortcut for Plex DupeFinder.
sudo ln -s /opt/plex_dupefinder/plex_dupefinder.py /usr/local/bin/plex_dupefinder
Generate a config.json
file.
plex_dupefinder
Fill in Plex URL and credentials at the prompt to generated a Plex Access Token (optional).
Dumping default config to: /opt/plex_dupefinder/config.json
Plex Server URL: http://localhost:32400
Plex Username: your_plex_username
Plex Password: your_plex_password
Auto Delete duplicates? [y/n]: n
Please edit the default configuration before running again!
Configure the config.json
file.
nano config.json
{
"AUDIO_CODEC_SCORES": {
"Unknown": 0,
"aac": 1000,
"ac3": 1000,
"dca": 2000,
"dca-ma": 4000,
"eac3": 1250,
"flac": 2500,
"mp2": 500,
"mp3": 1000,
"pcm": 2500,
"truehd": 4500,
"wmapro": 200
},
"AUTO_DELETE": false,
"FIND_DUPLICATE_FILEPATHS_ONLY": false,
"FILENAME_SCORES": {
"*.avi": -1000,
"*.ts": -1000,
"*.vob": -5000,
"*1080p*BluRay*": 15000,
"*720p*BluRay*": 10000,
"*HDTV*": -1000,
"*PROPER*": 1500,
"*REPACK*": 1500,
"*Remux*": 20000,
"*WEB*CasStudio*": 5000,
"*WEB*KINGS*": 5000,
"*WEB*NTB*": 5000,
"*WEB*QOQ*": 5000,
"*WEB*SiGMA*": 5000,
"*WEB*TBS*": -1000,
"*WEB*TROLLHD*": 2500,
"*WEB*VISUM*": 5000,
"*dvd*": -1000
},
"PLEX_LIBRARIES": [
"Movies",
"TV"
],
"PLEX_SERVER": "https://plex.your-server.com",
"PLEX_TOKEN": "",
"SCORE_FILESIZE": true,
"SKIP_LIST": [],
"VIDEO_CODEC_SCORES": {
"Unknown": 0,
"h264": 10000,
"h265": 5000,
"hevc": 5000,
"mpeg1video": 250,
"mpeg2video": 250,
"mpeg4": 500,
"msmpeg4": 100,
"msmpeg4v2": 100,
"msmpeg4v3": 100,
"vc1": 3000,
"vp9": 1000,
"wmv2": 250,
"wmv3": 250
},
"VIDEO_RESOLUTION_SCORES": {
"1080": 10000,
"480": 3000,
"4k": 20000,
"720": 5000,
"Unknown": 0,
"sd": 1000
}
}
The scoring is based on: non-configurable and configurable parameters.
Non-configurable parameters are: bitrate, duration, height, width, and audio channel.
Configurable parameters are: audio codec scores, video codec scores, video resolution scores, filename scores, and file sizes (can only be toggled on or off).
Note: bitrate, duration, height, width, audio channel, audio and video codecs, video resolutions (e.g. SD, 480p, 720p, 1080p, 4K, etc), and file sizes are all taken from the metadata Plex retrieves during media analysis.
You can set AUDIO_CODEC_SCORES
to your preference.
The default settings should be sufficient for most.
Under AUTO_DELETE
, set your desired option.
"AUTO_DELETE": true,
- Plex DupeFinder will run in automatic mode.
"AUTO_DELETE": false,
- Plex DupeFinder will run in interactive mode. (Default)
Options:
Skip (i.e. keep both): 0
Choose the best one (and delete the rest): b
Select the item to keep (and delete the rest): #
(i.e. 1
, 2
, 3
, etc).
Finds duplicates that only share the same file path.
"FIND_DUPLICATE_FILEPATHS_ONLY": false,
This option has a very limited use case, i.e. in instances where Plex may have glitched and created multiple duplicates of the same media item.
If using this setting, we recommend using UnionFS-Fuse that can generate whiteout files (*_HIDDEN~
) to prevent the deletion of the actual file on the system. The _HIDDEN~
files can then be removed afterwards or even during the dupe cleanup (e.g. watch -n 5 rm -rf /mnt/local/.unionfs-fuse/*
).
The default settings should be sufficient for most.
You can set FILENAME_SCORES
to your preference.
The default settings should be sufficient for most.
Go to Plex and get all the names of your Plex Libraries you want to find duplicates in.
Example Library:
Under PLEX_LIBRARIES
, list your Plex Libraries exactly as they are named in your Plex.
Format:
"PLEX_LIBRARIES": [
"LIBRARY_NAME_1",
"LIBRARY_NAME_2"
],
or
"PLEX_LIBRARIES": ["LIBRARY_NAME_1", "LIBRARY_NAME_2"],
Example:
"PLEX_LIBRARIES": [
"Movies",
"TV"
],
Your Plex server's URL.
This can be any format (e.g. http://localhost:32400, https://plex.domain.ltd).
Obtain a Plex Access Token:
Fill in the Plex URL and Plex login credentials, at the prompt, on first run. This only occurs when there is no config.json
present.
or
Visit https://support.plex.tv/hc/en-us/articles/204059436-Finding-an-authentication-token-X-Plex-Token
Add the Plex Access Token to "PLEX_TOKEN"
so that it now appears as "PLEX_TOKEN": "abcd1234",
.
"
) and there is a comma (,
) after it."SCORE_FILESIZE": true
will add more points to the overall score based on the actual file size.
The default settings should be sufficient for most.
Note: In some situations (e.g. a bad encode resulting in a large size), this may be something you want to turn it off (i.e. false
).
In Auto Delete mode, any file paths matching the patterns (i.e folders), listed in SKIP_LIST
, will be ignored.
Example:
"SKIP_LIST": ["/Movies4K/"]
The default settings should be sufficient for most.
You can set VIDEO_CODEC_SCORES
to your preference.
The default settings should be sufficient for most.
You can set VIDEO_RESOLUTION_SCORES
to your preference.
The default settings should be sufficient for most.
You will need to make sure that Allow media deletion is enabled in Plex.
In Plex, click the Settings icon -> Server -> Library.
Set the following:
enabled
Click SAVE CHANGES.
Simply run the script/command:
plex_dupefinder
If you find this project helpful, feel free to make a small donation to the developer: