Closed eikowagenknecht closed 4 years ago
b287c913d7070e3f97dd3730de9cbeca8001cf9f
New option detailed-progress
in the converter settings to show this
Wow, amazingly fast implementation, tank you very much! But it seems there is something wrong with it. Here is the console output of a typical conversion:
Processing /tv/xxx/Season 01/yyy - HDTV-1080p x265 10 [JA+EN] Opus [EN].mkv...
Manual processor started.
Python 64-bit 3.6.9 (default, Jul 17 2020, 12:50:27)
[GCC 8.4.0].
Guessit version: 3.1.1.
/usr/local/sma/venv/bin/python3
Loading config file ./autoProcessShows.ini.
Force conversion of files enabled. As a result conversion of mp4 files is also enabled
Unable to guess movie information
Traceback (most recent call last):
File "/usr/local/sma/manual.py", line 163, in guessInfo
return tvInfo(guess, tmdbid=tmdbid, tvdbid=tvdbid, imdbid=imdbid, season=season, episode=episode)
File "/usr/local/sma/manual.py", line 213, in tvInfo
metadata = Metadata(MediaType.TV, tmdbid=tmdbid, imdbid=imdbid, tvdbid=tvdbid, season=season, episode=episode, language=settings.taglanguage, logger=log)
File "/usr/local/sma/resources/metadata.py", line 94, in __init__
self.episodedata = episodequery.info(language=self.language)
File "/usr/local/sma/venv/lib/python3.6/site-packages/tmdbsimple/tv.py", line 646, in info
response = self._GET(path, kwargs)
File "/usr/local/sma/venv/lib/python3.6/site-packages/tmdbsimple/base.py", line 93, in _GET
return self._request('GET', path, params=params)
File "/usr/local/sma/venv/lib/python3.6/site-packages/tmdbsimple/base.py", line 88, in _request
response.raise_for_status()
File "/usr/local/sma/venv/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://api.themoviedb.org/3/tv/77199/season/1/episode/2?language=en&api_key=45e408d2851e968e6e4d0353ce621c66
Processing file /tv/xxx/Season 01/yyy - HDTV-1080p x265 10 [JA+EN] Opus [EN].mkv
Input Data
{
"format": "matroska,webm",
"format-fullname": "unknown",
"video": {
"index": 0,
"codec": "hevc",
"pix_fmt": "yuv420p10le",
"profile": "2",
"fps": 23.976023976023978,
"level": 12.0,
"field_order": "unknown"
},
"audio": [
{
"index": 1,
"codec": "opus",
"channels": 2,
"samplerate": 48000,
"language": "jpn",
"disposition": "-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired"
},
{
"index": 2,
"codec": "opus",
"channels": 2,
"samplerate": 48000,
"language": "eng",
"disposition": "+default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired"
}
],
"subtitle": [
{
"index": 3,
"codec": "ass",
"disposition": "+default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired",
"language": "eng"
},
{
"index": 4,
"codec": "hdmv_pgs_subtitle",
"disposition": "-default-dub-original-comment-lyrics-karaoke+forced-hearing_impaired-visual_impaired",
"language": "eng"
},
{
"index": 5,
"codec": "hdmv_pgs_subtitle",
"disposition": "-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired",
"language": "eng"
}
],
"attachment": [
{
"index": 6,
"codec": "ttf"
},
{
"index": 7,
"codec": "ttf"
},
{
"index": 8,
"codec": "otf"
},
{
"index": 9,
"codec": "otf"
},
{
"index": 10,
"codec": "ttf"
},
{
"index": 11,
"codec": "ttf"
},
{
"index": 12,
"codec": "ttf"
},
{
"index": 13,
"codec": "ttf"
},
{
"index": 14,
"codec": "ttf"
},
{
"index": 15,
"codec": "ttf"
},
{
"index": 16,
"codec": "ttf"
},
{
"index": 17,
"codec": "ttf"
},
{
"index": 18,
"codec": "ttf"
}
]
}
Reading video stream.
Video codec detected: hevc.
Pix Fmt: yuv420p10le.
Profile: 2.
Acceptable profile match found for VBR 1679.616 using CRF 23, maxrate 5m, bufsize 15m.
Video codec parameters None.
Creating h265vaapi video stream from source stream 0.
Reading audio streams.
Audio detected for stream 2 - opus eng 2 channel.
Creating aac audio stream from source stream 2.
Audio detected for stream 1 - opus jpn 2 channel.
Creating aac audio stream from source stream 1.
Default audio stream set to eng aac 2 channel stream [default-more-channels: True].
Reading subtitle streams.
Text-based subtitle detected for stream 3 - ass eng.
Creating mov_text subtitle stream from source stream 3.
Image-based subtitle detected for stream 4 - hdmv_pgs_subtitle eng.
Image-based subtitle detected for stream 5 - hdmv_pgs_subtitle eng.
Attempting to download subtitles.
Scanned for external subtitles and found 0 results in your approved languages.
vaapi hwaccel is supported by this ffmpeg build and will be used [hwaccels].
Output Data
{
"source": [
"/tv/xxx/Season 01/yyy - HDTV-1080p x265 10 [JA+EN] Opus [EN].mkv"
],
"format": "mp4",
"video": {
"codec": "h265vaapi",
"map": 0,
"bitrate": 1679.616,
"crf": 23,
"maxrate": "5m",
"bufsize": "15m",
"level": 0.0,
"profile": null,
"pix_fmt": "yuv420p",
"field_order": "unknown",
"width": null,
"filter": null,
"params": null,
"title": "FHD",
"debug": "video.pix_fmt"
},
"audio": [
{
"map": 2,
"codec": "aac",
"channels": 2,
"bitrate": 256,
"filter": null,
"samplerate": null,
"language": "eng",
"disposition": "+default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired",
"bsf": null,
"title": "Stereo",
"debug": "universal-audio"
},
{
"map": 1,
"codec": "aac",
"channels": 2,
"bitrate": 256,
"filter": null,
"samplerate": null,
"language": "jpn",
"disposition": "-default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired",
"bsf": null,
"title": "Stereo",
"debug": "universal-audio"
}
],
"subtitle": [
{
"map": 3,
"codec": "mov_text",
"language": "eng",
"encoding": "",
"disposition": "+default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired",
"title": "",
"debug": "subtitle.embed-subs"
}
],
"attachment": []
}
Preopts
[
"-fix_sub_duration",
"-hwaccel",
"vaapi"
]
Postopts
[
"-threads",
"0",
"-metadata:g",
"encoding_tool=SMA"
]
Downloaded Subtitles
[]
Starting conversion.
And from here on the new additions:
FFmpeg command:
======================
/usr/local/bin/ffmpeg -fix_sub_duration -hwaccel vaapi -i /tv/xxx/Season 01/yyy - HDTV-1080p x265 10 [JA+EN] Opus [EN].mkv -vcodec hevc_vaapi -map 0:0 -pix_fmt yuv420p -crf 23 -maxrate:v 5m -bufsize 15m -metadata:s:v BPS=1679000 -metadata:s:v BPS-eng=1679000 -metadata:s:v title=FHD -level 0.0 -tag:v hvc1 -vaapi_device /dev/dri/renderD128 -vf format=nv12,hwupload -c:a:0 aac -map 0:2 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0 BPS=256000 -metadata:s:a:0 BPS-eng=256000 -metadata:s:a:0 title=Stereo -metadata:s:a:0 language=eng -disposition:a:0 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:a:1 aac -map 0:1 -ac:a:1 2 -b:a:1 256k -metadata:s:a:1 BPS=256000 -metadata:s:a:1 BPS-eng=256000 -metadata:s:a:1 title=Stereo -metadata:s:a:1 language=jpn -disposition:a:1 -default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -strict experimental -c:s:0 mov_text -map 0:3 -metadata:s:s:0 title= -metadata:s:s:0 language=eng -disposition:s:0 +default-dub-original-comment-lyrics-karaoke-forced-hearing_impaired-visual_impaired-captions -f mp4 -threads 0 -metadata:g encoding_tool=SMA -y /tv/xxx/Season 01/yyy - HDTV-1080p x265 10 [JA+EN] Opus [EN].mp4
======================
[ ] 0% ffmpeg version N-98491-g3a37aa5 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
configuration: --disable-debug --disable-doc --disable-ffplay --enable-shared --enable-avresample --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-gpl --enable-libass --enable-fontconfig --enable-libfreetype --enable-libvidstab --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libx265 --enable-libxvid --enable-libx264 --enable-nonfree --enable-openssl --enable-libfdk_aac --enable-postproc --enable-small --enable-version3 --enable-libbluray --enable-libzmq --extra-libs=-ldl --prefix=/opt/ffmpeg --enable-libopenjpeg --enable-libkvazaar --enable-libaom --extra-libs=-lpthread --enable-libsrt --enable-vaapi --extra-cflags=-I/opt/ffmpeg/include --extra-ldflags=-L/opt/ffmpeg/lib
libavutil 56. 55.100 / 56. 55.100
libavcodec 58. 96.100 / 58. 96.100
libavformat 58. 48.100 / 58. 48.100
libavdevice 58. 11.101 / 58. 11.101
libavfilter 7. 87.100 / 7. 87.100
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 8.100 / 5. 8.100
libswresample 3. 8.100 / 3. 8.100
libpostproc 55. 8.100 / 55. 8.100
[matroska,webm @ 0x55f9b0cffa80] Could not find codec parameters for stream 4 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
[matroska,webm @ 0x55f9b0cffa80] Could not find codec parameters for stream 5 (Subtitle: hdmv_pgs_subtitle (pgssub)): unspecified size
Consider increasing the value for the 'analyzeduration' (0) and 'probesize' (5000000) options
Input #0, matroska,webm, from '/tv/xxx/Season 01/yyy - HDTV-1080p x265 10 [JA+EN] Opus [EN].mkv':
Metadata:
creation_time : 2019-10-20T07:25:12.000000Z
ENCODER : Lavf58.20.100
Duration: 00:23:43.20, start: 0.000000, bitrate: 1679 kb/s
Chapter #0:0: start 0.000000, end 40.832000
Metadata:
title : Chapter 1
Chapter #0:1: start 40.832000, end 132.591000
Metadata:
title : Chapter 2
Chapter #0:2: start 132.591000, end 712.545000
Metadata:
title : Chapter 3
Chapter #0:3: start 712.545000, end 1333.165000
Metadata:
title : Chapter 4
Chapter #0:4: start 1333.165000, end 1423.047000
Metadata:
title : Chapter 5
Stream #0:0: Video: hevc, yuv420p10le(tv, bt709), 1920x1080 [SAR 1:1 DAR 16:9], 23.98 fps, 23.98 tbr, 1k tbn, 23.98 tbc (default)
Metadata:
title : [Judas] x265 10b
BPS-eng : 1422396
DURATION-eng : 00:23:43.089000000
NUMBER_OF_FRAMES-eng: 34115
NUMBER_OF_BYTES-eng: 253024663
_STATISTICS_WRITING_APP-eng: mkvmerge v38.0.0 ('The Silent Type') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-10-20 07:25:12
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:1(jpn): Audio: opus, 48000 Hz, stereo, fltp
Metadata:
title : [Judas] Stereo (Opus)
BPS-eng : 108458
DURATION-eng : 00:23:43.180000000
NUMBER_OF_FRAMES-eng: 71159
NUMBER_OF_BYTES-eng: 19294448
_STATISTICS_WRITING_APP-eng: mkvmerge v38.0.0 ('The Silent Type') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-10-20 07:25:12
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:2(eng): Audio: opus, 48000 Hz, stereo, fltp (default)
Metadata:
title : [Judas] Stereo (Opus)
BPS-eng : 107771
DURATION-eng : 00:23:43.060000000
NUMBER_OF_FRAMES-eng: 71153
NUMBER_OF_BYTES-eng: 19170725
_STATISTICS_WRITING_APP-eng: mkvmerge v38.0.0 ('The Silent Type') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-10-20 07:25:12
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:3(eng): Subtitle: ass (default)
Metadata:
title : English[Full]
BPS-eng : 250
DURATION-eng : 00:23:27.910000000
NUMBER_OF_FRAMES-eng: 504
NUMBER_OF_BYTES-eng: 44110
_STATISTICS_WRITING_APP-eng: mkvmerge v38.0.0 ('The Silent Type') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-10-20 07:25:12
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:4(eng): Subtitle: hdmv_pgs_subtitle
Metadata:
title : English[Forced] (PGS)
BPS-eng : 5520
DURATION-eng : 00:23:25.696000000
NUMBER_OF_FRAMES-eng: 68
NUMBER_OF_BYTES-eng: 970053
_STATISTICS_WRITING_APP-eng: mkvmerge v38.0.0 ('The Silent Type') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-10-20 07:25:12
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:5(eng): Subtitle: hdmv_pgs_subtitle
Metadata:
title : English[Full] (PGS)
BPS-eng : 28323
DURATION-eng : 00:23:27.907000000
NUMBER_OF_FRAMES-eng: 484
NUMBER_OF_BYTES-eng: 4984537
_STATISTICS_WRITING_APP-eng: mkvmerge v38.0.0 ('The Silent Type') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-10-20 07:25:12
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
Stream #0:6: Attachment: ttf
Metadata:
filename : Champagne & Limousines.ttf
mimetype : application/x-truetype-font
Stream #0:7: Attachment: ttf
Metadata:
filename : Comfortaa-Regular.ttf
mimetype : application/x-truetype-font
Stream #0:8: Attachment: otf
Metadata:
filename : FONTIN_SANS.OTF
mimetype : application/vnd.ms-opentype
Stream #0:9: Attachment: otf
Metadata:
filename : FONTIN_SANS_BI.OTF
mimetype : application/vnd.ms-opentype
Stream #0:10: Attachment: ttf
Metadata:
filename : Gabriola.ttf
mimetype : application/x-truetype-font
Stream #0:11: Attachment: ttf
Metadata:
filename : HollaScript.ttf
mimetype : application/x-truetype-font
Stream #0:12: Attachment: ttf
Metadata:
filename : LinotypeFinneganMedium.ttf
mimetype : application/x-truetype-font
Stream #0:13: Attachment: ttf
Metadata:
filename : LT_1.TTF
mimetype : application/x-truetype-font
Stream #0:14: Attachment: ttf
Metadata:
filename : Material_Sans.ttf
mimetype : application/x-truetype-font
Stream #0:15: Attachment: ttf
Metadata:
filename : raavi.ttf
mimetype : application/x-truetype-font
Stream #0:16: Attachment: ttf
Metadata:
filename : raavib.ttf
mimetype : application/x-truetype-font
Stream #0:17: Attachment: ttf
Metadata:
filename : SAO_Sign.ttf
mimetype : application/x-truetype-font
Stream #0:18: Attachment: ttf
Metadata:
filename : SAO_Sign_1.ttf
mimetype : application/x-truetype-font
Codec AVOption crf (Select the quality for constant quality mode) specified for output file #0 (/tv/xxx/Season 01/yyy - HDTV-1080p x265 10 [JA+EN] Opus [EN].mp4) has not been used for any stream. The most likely reason is either wrong type (e.g. a video option with no video streams) or that it is a private option of some encoder which was not actually used for any stream.
Stream mapping:
Stream #0:0 -> #0:0 (hevc (native) -> hevc (hevc_vaapi))
Stream #0:2 -> #0:1 (opus (native) -> aac (native))
Stream #0:1 -> #0:2 (opus (native) -> aac (native))
Stream #0:3 -> #0:3 (ass (ssa) -> mov_text (native))
Press [q] to stop, [?] for help
Incompatible pixel format 'yuv420p' for codec 'hevc_vaapi', auto-selecting format 'vaapi_vld'
[hevc_vaapi @ 0x55f9b0ea5c80] No quality level set; using default (25).
Output #0, mp4, to '/tv/xxx/Season 01/yyy - HDTV-1080p x265 10 [JA+EN] Opus [EN].mp4':
Metadata:
encoding_tool : SMA
encoder : Lavf58.48.100
Chapter #0:0: start 0.000000, end 40.832000
Metadata:
title : Chapter 1
Chapter #0:1: start 40.832000, end 132.591000
Metadata:
title : Chapter 2
Chapter #0:2: start 132.591000, end 712.545000
Metadata:
title : Chapter 3
Chapter #0:3: start 712.545000, end 1333.165000
Metadata:
title : Chapter 4
Chapter #0:4: start 1333.165000, end 1423.047000
Metadata:
title : Chapter 5
Stream #0:0: Video: hevc (hevc_vaapi) (hvc1 / 0x31637668), vaapi_vld, 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 23.98 fps, 24k tbn, 23.98 tbc (default)
Metadata:
BPS-eng : 1679000
BPS : 1679000
DURATION-eng : 00:23:43.089000000
NUMBER_OF_FRAMES-eng: 34115
NUMBER_OF_BYTES-eng: 253024663
_STATISTICS_WRITING_APP-eng: mkvmerge v38.0.0 ('The Silent Type') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-10-20 07:25:12
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
title : FHD
encoder : Lavc58.96.100 hevc_vaapi
Stream #0:1(eng): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 256 kb/s (default)
Metadata:
title : Stereo
BPS-eng : 256000
BPS : 256000
DURATION-eng : 00:23:43.060000000
NUMBER_OF_FRAMES-eng: 71153
NUMBER_OF_BYTES-eng: 19170725
_STATISTICS_WRITING_APP-eng: mkvmerge v38.0.0 ('The Silent Type') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-10-20 07:25:12
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
encoder : Lavc58.96.100 aac
Stream #0:2(jpn): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 256 kb/s
Metadata:
title : Stereo
BPS-eng : 256000
BPS : 256000
DURATION-eng : 00:23:43.180000000
NUMBER_OF_FRAMES-eng: 71159
NUMBER_OF_BYTES-eng: 19294448
_STATISTICS_WRITING_APP-eng: mkvmerge v38.0.0 ('The Silent Type') 64-bit
_STATISTICS_WRITING_DATE_UTC-eng: 2019-10-20 07:25:12
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
encoder : Lavc58.96.100 aac
Stream #0:3(eng): Subtitle: mov_text (tx3g / 0x67337874) (default)
Metadata:
_STATISTICS_WRITING_DATE_UTC-eng: 2019-10-20 07:25:12
_STATISTICS_TAGS-eng: BPS DURATION NUMBER_OF_FRAMES NUMBER_OF_BYTES
BPS-eng : 250
DURATION-eng : 00:23:27.910000000
NUMBER_OF_FRAMES-eng: 504
NUMBER_OF_BYTES-eng: 44110
_STATISTICS_WRITING_APP-eng: mkvmerge v38.0.0 ('The Silent Type') 64-bit
encoder : Lavc58.96.100 mov_text
[ ] 0% frame= 18 fps= 17 q=-0.0 size= 0kB time=00:00:01.12 bitrate= 1.1kbits/s dup=3 drop=0 speed=1.
[ ] 0% frame= 27 fps= 17 q=-0.0 size= 0kB time=00:00:01.44 bitrate= 0.9kbits/s dup=3 drop=0 speed=0.
[ ] 0% frame= 38 fps= 18 q=-0.0 size= 0kB time=00:00:01.91 bitrate= 0.7kbits/s dup=3 drop=0 speed=0.
[ ] 0% frame= 47 fps= 18 q=-0.0 size= 0kB time=00:00:02.23 bitrate= 0.6kbits/s dup=3 drop=0 speed=0.
[ ] 0% frame= 58 fps= 18 q=-0.0 size= 0kB time=00:00:02.72 bitrate= 0.5kbits/s dup=3 drop=0 speed=0.
[ ] 0% frame= 68 fps= 18 q=-0.0 size= 0kB time=00:00:03.32 bitrate= 0.4kbits/s dup=3 drop=0 speed=0.
[ ] 0% frame= 78 fps= 18 q=-0.0 size= 0kB time=00:00:03.68 bitrate= 0.3kbits/s dup=3 drop=0 speed=0.
[ ] 0% frame= 88 fps= 18 q=-0.0 size= 0kB time=00:00:04.11 bitrate= 0.3kbits/s dup=3 drop=0 speed=0.
[ ] 0% frame= 98 fps= 18 q=-0.0 size= 0kB time=00:00:04.47 bitrate= 0.3kbits/s dup=3 drop=0 speed=0.
[ ] 0% frame= 108 fps= 18 q=-0.0 size= 0kB time=00:00:04.96 bitrate= 0.3kbits/s dup=3 drop=0 speed=0.849x
I see two things here:
1) After the first progress [ ] 0%
the whole ffmpeg output is written. While very informative, this seems to be a bit too much. Is this on purpose?
2) Depending on the terminal width, the last line repeats itself every second or so, instead of being replaced. I think this could be because it is ~133 characters long, wider than a usual terminal window. I don't know if this can even be fixed without shortening the informations.
Thank you again! If I find a solution to the line writing, I'll create a PR. Until then I will just resize the terminal so >140characters when running your script.
I've been playing with varios settings regarding quality and encoders recently. It would help me quite a bit to see more information next to the progress bar.
Currently it only shows:
When the ffmpeg command is run directly there ist much more info while encoding, like:
Maybe you could show that as well, like:
I'd especially like the "speed" and "frame" values. Speed because it shows if hardware accelleration is working correctly and Frame because it helps me see when the process is hung up (some files just stop at a certain percentage and hand there for days if I do not interrupt manually).