Wrong thumbnail in mp3 file when downloading audio from videos with identical names

bersbersbers commented 4 years ago

Checklist

[x] I'm reporting a broken site support issue
[x] I've verified that I'm running youtube-dl version 2020.06.16.1
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar bug reports including closed ones
[x] I've read bugs section in FAQ

I have installed https://github.com/ytdl-org/youtube-dl/pull/25717 through pip install git+https://github.com/ytdl-org/youtube-dl.git@refs/pull/25717/head

Verbose log

Commands: ``` rem This is on Windows set RANDOM=152252588 md D:\Temp pushd D:\Temp rem Delete everything del /q "D:\Temp\*.*" rem Download yJb3pszkp3o (the red one) youtube-dl -v yJb3pszkp3o --format bestaudio --keep-video --write-thumbnail --add-metadata --extract-audio --audio-format mp3 --embed-thumbnail --output "%RANDOM%-%(title)s.%(ext)s" rem Delete everything del /q "D:\Temp\*.*" rem Download es2BBkkKN7Q (the green one) youtube-dl -v es2BBkkKN7Q --format bestaudio --keep-video --write-thumbnail --add-metadata --extract-audio --audio-format mp3 --embed-thumbnail --output "%RANDOM%-%(title)s.%(ext)s" rem Now you have a jpg file with the logo of es2BBkkKN7Q, and an mp3 file with the logo of yJb3pszkp3o (the red one) "%RANDOM%-Kai Castro - Incoherence (Austin Pettit Remix).jpg" "%RANDOM%-Kai Castro - Incoherence (Austin Pettit Remix).mp3" ``` Output: ``` D:\Temp>rem This is on Windows D:\Temp>set RANDOM=152252588 D:\Temp> D:\Temp>md D:\Temp A subdirectory or file D:\Temp already exists. D:\Temp>pushd D:\Temp D:\Temp> D:\Temp>rem Delete everything D:\Temp>del /q "D:\Temp\*.*" D:\Temp> D:\Temp>rem Download yJb3pszkp3o (the red one) D:\Temp>youtube-dl -v yJb3pszkp3o --format bestaudio --keep-video --write-thumbnail --add-metadata --extract-audio --audio-format mp3 --embed-thumbnail --output "%RANDOM%-%(title)s.%(ext)s" [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['-v', 'yJb3pszkp3o', '--format', 'bestaudio', '--keep-video', '--write-thumbnail', '--add-metadata', '--extract-audio', '--audio-format', 'mp3', '--embed-thumbnail', '--output', '152252588-%(title)s.%(ext)s'] [debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252 [debug] youtube-dl version 2020.06.16.1 [debug] Python version 3.8.3 (CPython) - Windows-10-10.0.19041-SP0 [debug] exe versions: ffmpeg 4.3, ffprobe 4.3, rtmpdump 2.2d [debug] Proxy map: {} [youtube] yJb3pszkp3o: Downloading webpage [youtube] {18} signature length 106, html5 player 3662280c [youtube] {136} signature length 102, html5 player 3662280c [youtube] {247} signature length 106, html5 player 3662280c [youtube] {135} signature length 106, html5 player 3662280c [youtube] {244} signature length 106, html5 player 3662280c [youtube] {134} signature length 102, html5 player 3662280c [youtube] {243} signature length 102, html5 player 3662280c [youtube] {133} signature length 106, html5 player 3662280c [youtube] {242} signature length 106, html5 player 3662280c [youtube] {160} signature length 106, html5 player 3662280c [youtube] {278} signature length 102, html5 player 3662280c [youtube] {140} signature length 106, html5 player 3662280c [youtube] {251} signature length 106, html5 player 3662280c [youtube] yJb3pszkp3o: Downloading MPD manifest [youtube] yJb3pszkp3o: Downloading thumbnail ... [youtube] yJb3pszkp3o: Writing thumbnail to: 152252588-Kai Castro - Incoherence (Austin Pettit Remix).webp [debug] Invoking downloader on 'https://manifest.googlevideo.com/api/manifest/dash/expire/1594649882/ei/uhgMX6CSMPLI8gOw45jIBQ/ip/79.212.72.172/id/c896f7a6cce4a77a/source/youtube/requiressl/yes/playback_host/r3---sn-4g5e6nzz.googlevideo.com/mh/hr/mm/31%2C29/mn/sn-4g5e6nzz%2Csn-4g5ednee/ms/au%2Crdu/mv/m/mvi/3/pl/26/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/initcwndbps/1306250/vprv/1/mt/1594628194/fvip/6/keepalive/yes/itag/0/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Chfr%2Cas%2Cvprv%2Citag/sig/AOq0QJ8wRgIhAPqkiRLFqNZ3HD6iMrA71DaGn7Tooz7h04WiICXqf7EKAiEAsCn0NpyjD0HlbG0WZNJwweorPBUPrWry1gK3_jYt01E%3D/lsparams/playback_host%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps/lsig/AG3C_xAwRAIgNOarzbSMPgHIZ6iZtR30qFDGVwHU4u-Hs8uZkMUkJrsCIFpdi1Zubdx0Ypt4_XfRGsbH4331tOlvOOfh3EzClTh0' [dashsegments] Total fragments: 39 [download] Destination: 152252588-Kai Castro - Incoherence (Austin Pettit Remix).webm [download] 100% of 6.13MiB in 00:04 [debug] ffmpeg command line: ffprobe -show_streams "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).webm" [ffmpeg] Destination: 152252588-Kai Castro - Incoherence (Austin Pettit Remix).mp3 [debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).webm" -vn -acodec libmp3lame "-q:a" 5 "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).mp3" [ffmpeg] Adding metadata to '152252588-Kai Castro - Incoherence (Austin Pettit Remix).mp3' [debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).mp3" -c copy -metadata "title=Kai Castro - Incoherence (Austin Pettit Remix)" -metadata "date=20181025" -metadata "description=I do not claim to own the copyright to this music. This upload is only for promotional purposes." -metadata "comment=I do not claim to own the copyright to this music. This upload is only for promotional purposes." -metadata "purl=https://www.youtube.com/watch?v=yJb3pszkp3o" -metadata "artist=Michael K." "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).temp.mp3" [ffmpeg] Converting thumbnail "152252588-Kai Castro - Incoherence (Austin Pettit Remix).webp" to JPEG [debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).webp" "-bsf:v" mjpeg2jpeg "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).jpg" [ffmpeg] Adding thumbnail to "152252588-Kai Castro - Incoherence (Austin Pettit Remix).mp3" [debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).mp3" -i "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).jpg" -c copy -map 0 -map 1 "-metadata:s:v" "title=\"Album cover\"" "-metadata:s:v" "comment=\"Cover (Front)\"" "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).temp.mp3" D:\Temp> D:\Temp>rem Delete everything D:\Temp>del /q "D:\Temp\*.*" D:\Temp> D:\Temp>rem Download es2BBkkKN7Q (the green one) D:\Temp>youtube-dl -v es2BBkkKN7Q --format bestaudio --keep-video --write-thumbnail --add-metadata --extract-audio --audio-format mp3 --embed-thumbnail --output "%RANDOM%-%(title)s.%(ext)s" [debug] System config: [] [debug] User config: [] [debug] Custom config: [] [debug] Command-line args: ['-v', 'es2BBkkKN7Q', '--format', 'bestaudio', '--keep-video', '--write-thumbnail', '--add-metadata', '--extract-audio', '--audio-format', 'mp3', '--embed-thumbnail', '--output', '152252588-%(title)s.%(ext)s'] [debug] Encodings: locale cp1252, fs utf-8, out utf-8, pref cp1252 [debug] youtube-dl version 2020.06.16.1 [debug] Python version 3.8.3 (CPython) - Windows-10-10.0.19041-SP0 [debug] exe versions: ffmpeg 4.3, ffprobe 4.3, rtmpdump 2.2d [debug] Proxy map: {} [youtube] es2BBkkKN7Q: Downloading webpage [youtube] {18} signature length 106, html5 player 3662280c [youtube] {136} signature length 106, html5 player 3662280c [youtube] {247} signature length 106, html5 player 3662280c [youtube] {135} signature length 106, html5 player 3662280c [youtube] {244} signature length 102, html5 player 3662280c [youtube] {134} signature length 106, html5 player 3662280c [youtube] {243} signature length 106, html5 player 3662280c [youtube] {133} signature length 106, html5 player 3662280c [youtube] {242} signature length 106, html5 player 3662280c [youtube] {160} signature length 102, html5 player 3662280c [youtube] {278} signature length 106, html5 player 3662280c [youtube] {140} signature length 106, html5 player 3662280c [youtube] {251} signature length 102, html5 player 3662280c [youtube] es2BBkkKN7Q: Downloading MPD manifest [youtube] es2BBkkKN7Q: Downloading thumbnail ... [youtube] es2BBkkKN7Q: Writing thumbnail to: 152252588-Kai Castro - Incoherence (Austin Pettit Remix).webp [debug] Invoking downloader on 'https://manifest.googlevideo.com/api/manifest/dash/expire/1594649898/ei/yhgMX--cNIXZ1wLqrYG4Bw/ip/79.212.72.172/id/7acd8106490a37b4/source/youtube/requiressl/yes/playback_host/r5---sn-4g5ednek.googlevideo.com/mh/gz/mm/31%2C29/mn/sn-4g5ednek%2Csn-i5heen7z/ms/au%2Crdu/mv/m/mvi/5/pl/26/hfr/all/as/fmp4_audio_clear%2Cwebm_audio_clear%2Cwebm2_audio_clear%2Cfmp4_sd_hd_clear%2Cwebm2_sd_hd_clear/initcwndbps/1278750/vprv/1/mt/1594628255/fvip/5/keepalive/yes/itag/0/sparams/expire%2Cei%2Cip%2Cid%2Csource%2Crequiressl%2Chfr%2Cas%2Cvprv%2Citag/sig/AOq0QJ8wRAIgRZA5C_wbztMWFu1iG6dz_S6e1_Ma1vGqN-k2xD8V2DACIHeu0Rt8Bt1kAp7VI9ltC7v_13aaOmQ9luJ4IPCK_0TO/lsparams/playback_host%2Cmh%2Cmm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps/lsig/AG3C_xAwRAIgAun3fXEPHVfOkK_BAmR9M70a6NOe0dOnVmffikQ8IlgCIDmpGOhBCVV9menSbFiPF1KYzTfYSjcvIcAmU7OxJAxR' [dashsegments] Total fragments: 39 [download] Destination: 152252588-Kai Castro - Incoherence (Austin Pettit Remix).webm [download] 100% of 6.17MiB in 00:04 [debug] ffmpeg command line: ffprobe -show_streams "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).webm" [ffmpeg] Destination: 152252588-Kai Castro - Incoherence (Austin Pettit Remix).mp3 [debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).webm" -vn -acodec libmp3lame "-q:a" 5 "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).mp3" [ffmpeg] Adding metadata to '152252588-Kai Castro - Incoherence (Austin Pettit Remix).mp3' [debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).mp3" -c copy -metadata "title=Kai Castro - Incoherence (Austin Pettit Remix)" -metadata "date=20181120" -metadata "description=Kai Castro - Incoherence EP (Mystic Carousel Records) Oct 22, 2018 Grap your Copy: https://www.beatport.com/release/incoherence/2292615 Style: Prog-House Cat..." -metadata "comment=Kai Castro - Incoherence EP (Mystic Carousel Records) Oct 22, 2018 Grap your Copy: https://www.beatport.com/release/incoherence/2292615 Style: Prog-House Cat..." -metadata "purl=https://www.youtube.com/watch?v=es2BBkkKN7Q" -metadata "artist=Mystic Carousel & Estribo Records" "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).temp.mp3" [ffmpeg] Converting thumbnail "152252588-Kai Castro - Incoherence (Austin Pettit Remix).webp" to JPEG [debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).webp" "-bsf:v" mjpeg2jpeg "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).jpg" [ffmpeg] Adding thumbnail to "152252588-Kai Castro - Incoherence (Austin Pettit Remix).mp3" [debug] ffmpeg command line: ffmpeg -y -loglevel "repeat+info" -i "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).mp3" -i "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).jpg" -c copy -map 0 -map 1 "-metadata:s:v" "title=\"Album cover\"" "-metadata:s:v" "comment=\"Cover (Front)\"" "file:152252588-Kai Castro - Incoherence (Austin Pettit Remix).temp.mp3" D:\Temp> D:\Temp>rem Now you have a jpg file with the logo of es2BBkkKN7Q, and an mp3 file with the logo of yJb3pszkp3o (the red one) D:\Temp>"%RANDOM%-Kai Castro - Incoherence (Austin Pettit Remix).jpg" D:\Temp>"%RANDOM%-Kai Castro - Incoherence (Austin Pettit Remix).mp3" D:\Temp> ```

Description

I noticed that when you download two videos with the same title, some content of the first video (in particular, the thumbnail) is integrated into the second video although I delete all files after the first download.

alexmerkel commented 4 years ago

I can't reproduce the issue on macOS.

When I run the first command, an MP3 with a red thumbnail and a red JPG are created. Following this up with the second command, an MP3 with a green/grey thumbnail and a green/grey JPG are created. I do not observe the mix-up behavior you describe.

bersbersbers commented 4 years ago

Hmm, that is unfortunate. I am willing to track this down further if anyone is interested. A few thoughts:

I wonder if this is related to any kind of caching on ffmpeg's side, which may be different on Windows and macOS. By the way, I am using ffmpeg version 4.3 Copyright (c) 2000-2020 the FFmpeg developers
I can still repro it using the code above.
I can not repro it using a different pair of YouTube videos with identical titels.
I can not repro it after removing --add-metadata, so that certainly plays a role, although I am not sure which one.
I can not repro if after removing the "collision" of file names resulting from the --output parameter.

bersbersbers commented 4 years ago

I believe VLC player may be tricking me. Opening the .mp3 file (even a renamed copy of it) in VLC shows the incorrect, red album art. Android music player and ffmpeg.exe are able to extract, from the same .mp3 file, the correct green album art. Maybe VLC is the cache I am looking for. (I should note that "Allow metadata network access" is not checked in VLC.)

That's it! %APPDATA%\vlc\art\arturl had a bunch of folders, and the 0d45412115229f5b87db051cbe028d27 one had the red art.jpg. Somehow, VLC must have picked up this hash (probably from the meta data, hence the importance of --add-metadata) and associated it to this album art, kind of forever. This is not unheard of. For sure, it's not youtube-dl then.

ytdl-org / youtube-dl