[Workaround Request] for the old AtomicParsley (included in latest debian/ubuntu) that cannot embed JPEG thumbnails that are not JFIF

sebma commented 3 years ago

Checklist

[x] I'm reporting a broken site support issue
[x] I've verified that I'm running youtube-dl version 2021.06.06
[x] I've checked that all provided URLs are alive and playable in a browser
[x] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[x] I've searched the bugtracker for similar bug reports including closed ones
[x] I've read bugs section in FAQ

Verbose log

$ youtube-dl --ignore-config --embed-thumbnail -v https://www.dailymotion.com/video/x2ehr3o
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ignore-config', '--embed-thumbnail', '-v', 'https://www.dailymotion.com/video/x2ehr3o']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.06.06
[debug] Python version 3.8.10 (CPython) - Linux-5.4.0-77-generic-x86_64-with-glibc2.29
[debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4, rtmpdump 2.4
[debug] Proxy map: {}
[dailymotion] Downloading Access Token
[dailymotion] x2ehr3o: Downloading media JSON metadata
[dailymotion] x2ehr3o: Downloading metadata JSON
[dailymotion] x2ehr3o: Downloading m3u8 information
[debug] Default format spec: bestvideo+bestaudio/best
[dailymotion] x2ehr3o: Downloading thumbnail ...
[dailymotion] x2ehr3o: Writing thumbnail to: L'esclavage moderne-x2ehr3o.jpg
[debug] Invoking downloader on 'https://proxy-034.dc3.dailymotion.com/sec(z1LdIxCmoIMisR-U4IapUaEcgnLOEo2HA01OqdyJmwybOePxVY4hKRNWRBGg1mIO7tSS0Dj4s3S7dSyXyXwmiw)/video/252/572/145275252_mp4_h264_aac_hq_2.mp4'
[download] Destination: L'esclavage moderne-x2ehr3o.mp4
[download] 100% of 117.59MiB in 16:01
[atomicparsley] Adding thumbnail to "L'esclavage moderne-x2ehr3o.mp4"
[debug] AtomicParsley command line: AtomicParsley 'L'"'"'esclavage moderne-x2ehr3o.mp4' --artwork 'L'"'"'esclavage moderne-x2ehr3o.jpg' -o 'L'"'"'esclavage moderne-x2ehr3o.temp.mp4'
ERROR: 
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 2115, in post_process
    files_to_delete, info = pp.run(info)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/postprocessor/embedthumbnail.py", line 116, in run
    raise EmbedThumbnailPPError(msg)
youtube_dl.postprocessor.embedthumbnail.EmbedThumbnailPPError

Description

The AtomicParsley tool supplied in the latest release of debian and ubuntu, which is still at v0.9.6-2 does not support JPEG thumbnails that are not JFIF :

$ file "L'esclavage moderne-x2ehr3o.jpg" 
L'esclavage moderne-x2ehr3o.jpg: JPEG image data, baseline, precision 8, 1884x1080, components 3

Can you please convert the downloaded thumbnails to JPEG/JFIF with convert of ffmpeg before actually calling AtomicParsley.

N.B. : I tried download and convert the thumbnail myself to JPEG/JFIF BEFORE calling youtube-dl, unfortunately youtube-dl re-downloads it and youtube-dl overwrites my newly converted thumbnail to the not JFIF format supplied by Dailymotion :disappointed: (the timestamp and the format of the downloaded thumbnail before are after running youtube-dl is not the same) :

$ wget -O "$(youtube-dl --ignore-config --get-filename -o "%(title)s-%(id)s" https://www.dailymotion.com/video/x2ehr3o).jpg" $(youtube-dl --ignore-config --get-thumbnail https://www.dailymotion.com/video/x2ehr3o)
2021-07-15 21:57:29 URL:https://s1.dmcdn.net/v/8gBbq1VtIDlcffqS6/x1080 [125508/125508] -> "L'esclavage moderne-x2ehr3o.jpg" [1]
$ file "L'esclavage moderne-x2ehr3o.jpg"
L'esclavage moderne-x2ehr3o.jpg: JPEG image data, baseline, precision 8, 1884x1080, components 3
$ convert "L'esclavage moderne-x2ehr3o.jpg" "L'esclavage moderne-x2ehr3o.NEW.jpg"
$ mv -vf "L'esclavage moderne-x2ehr3o.NEW.jpg" "L'esclavage moderne-x2ehr3o.jpg"
renamed "L'esclavage moderne-x2ehr3o.NEW.jpg" -> "L'esclavage moderne-x2ehr3o.jpg"
$ file "L'esclavage moderne-x2ehr3o.jpg" 
L'esclavage moderne-x2ehr3o.jpg: JPEG image data, JFIF standard 1.01, aspect ratio, density 1x1, segment length 16, baseline, precision 8, 1884x1080, components 3
$ ls -l --time-style=+'%Y-%m-%d %T' "L'esclavage moderne-x2ehr3o.jpg"
-rw-rw-r-- 1 sebastien sebastien 125508 2021-07-15 21:57:30 "L'esclavage moderne-x2ehr3o.jpg"
$ youtube-dl --ignore-config --embed-thumbnail -v -f hls-380-0 https://www.dailymotion.com/video/x2ehr3o
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['--ignore-config', '--embed-thumbnail', '-v', '-f', 'hls-380-0', 'https://www.dailymotion.com/video/x2ehr3o']
[debug] Encodings: locale UTF-8, fs utf-8, out utf-8, pref UTF-8
[debug] youtube-dl version 2021.06.06
[debug] Python version 3.8.10 (CPython) - Linux-5.4.0-77-generic-x86_64-with-glibc2.29
[debug] exe versions: ffmpeg 4.2.4, ffprobe 4.2.4, rtmpdump 2.4
[debug] Proxy map: {}
[dailymotion] Downloading Access Token
[dailymotion] x2ehr3o: Downloading media JSON metadata
[dailymotion] x2ehr3o: Downloading metadata JSON
[dailymotion] x2ehr3o: Downloading m3u8 information
[dailymotion] x2ehr3o: Downloading thumbnail ...
[dailymotion] x2ehr3o: Writing thumbnail to: L'esclavage moderne-x2ehr3o.jpg
[debug] Invoking downloader on 'https://proxy-006.ix7.dailymotion.com/sec(wei31X4-J0DHtJ8pD26EwzFNkyWeddSHSDHhPao91q_8Ifhzzxc-oUu9pPAoUh360wyFbyHmW2tk1NpZTjdYYqmgKGHleLrBOnmLGQWWH5U)/video/252/572/145275252_mp4_h264_aac_2.m3u8'
[hlsnative] Downloading m3u8 manifest
[hlsnative] Total fragments: 124
[download] Destination: L'esclavage moderne-x2ehr3o.mp4
[download] 100% of 72.95MiB in 00:14
[debug] ffmpeg command line: ffprobe -show_streams 'file:L'"'"'esclavage moderne-x2ehr3o.mp4'
[ffmpeg] Fixing malformed AAC bitstream in "L'esclavage moderne-x2ehr3o.mp4"
[debug] ffmpeg command line: ffmpeg -y -loglevel repeat+info -i 'file:L'"'"'esclavage moderne-x2ehr3o.mp4' -c copy -f mp4 -bsf:a aac_adtstoasc 'file:L'"'"'esclavage moderne-x2ehr3o.temp.mp4'
[atomicparsley] Adding thumbnail to "L'esclavage moderne-x2ehr3o.mp4"
[debug] AtomicParsley command line: AtomicParsley 'L'"'"'esclavage moderne-x2ehr3o.mp4' --artwork 'L'"'"'esclavage moderne-x2ehr3o.jpg' -o 'L'"'"'esclavage moderne-x2ehr3o.temp.mp4'
ERROR: 
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/YoutubeDL.py", line 2115, in post_process
    files_to_delete, info = pp.run(info)
  File "/usr/local/lib/python3.8/dist-packages/youtube_dl/postprocessor/embedthumbnail.py", line 116, in run
    raise EmbedThumbnailPPError(msg)
youtube_dl.postprocessor.embedthumbnail.EmbedThumbnailPPError

$ ls -l --time-style=+'%Y-%m-%d %T' "L'esclavage moderne-x2ehr3o.jpg"
-rw-rw-r-- 1 sebastien sebastien 125508 2021-07-15 21:57:32 "L'esclavage moderne-x2ehr3o.jpg"
$ file "L'esclavage moderne-x2ehr3o.jpg"
L'esclavage moderne-x2ehr3o.jpg: JPEG image data, baseline, precision 8, 1884x1080, components 3

ghost commented 3 years ago

Although AtomicParsley hasn't been updated for years, why not filing a bug report to AtomicParsley and/or asking Debian/Ubuntu package managers to include forked version of AtomicParsley? Besides, you know the forked version well.

I feel this request is strange, even if changing youtube-dl is not difficult.

sebma commented 3 years ago

@kikuyan I understand but in the meantime that debian/ubuntu update their atomicparsley package into their repositories, can please prevent youtube-dl from overwriting the (the manually pre-downloaded) thumbnail to the filesystem in the case where the video has never been downloaded by youtube-dl ?

BTW : What is the procedure to ask Debian/Ubuntu package managers to include forked version of AtomicParsley ?

ytdl-org / youtube-dl