search

mdhiggins / sickbeard_mp4_automator

Automatically convert video files to a standardized format with metadata tagging to create a beautiful and uniform media library
MIT License
1.53k stars 202 forks source link

Issue regarding the embedding of subtitles #1359

Closed dramirvf closed 3 years ago

dramirvf commented 3 years ago

I have a video file in MKV Format with the following information:

General
Unique ID                                : 141713911095963505082473410969543873118 (0x6A9D1B43087C65CA09F87D670317925E)
Complete name                            : Z:\Test - Test\Tag\Tag 2\2A - 1V - 5T\Source.mkv
Format                                   : Matroska
Format version                           : Version 4
File size                                : 33.8 MiB
Duration                                 : 2 min 1 s
Overall bit rate                         : 2 340 kb/s
Encoded date                             : UTC 2020-11-04 17:02:47
Writing application                      : mkvmerge v50.0.0 ('Awakenings') 64-bit
Writing library                          : libebml v1.4.0 + libmatroska v1.6.2
Cover                                    : Yes
Attachments                              : cover.jpg

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L4@Main
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 2 min 0 s
Bit rate                                 : 1 821 kb/s
Width                                    : 1 920 pixels
Height                                   : 800 pixels
Display aspect ratio                     : 2.40:1
Frame rate mode                          : Constant
Frame rate                               : 23.976 (23976/1000) FPS
Original frame rate                      : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.049
Stream size                              : 26.1 MiB (77%)
Writing library                          : x265 3.3+4-rarbg-30eb4de83092:[Linux][GCC 8.3.1][64 bit] 10bit
Encoding settings                        : cpuid=1111039 / frame-threads=4 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=1920x800 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=4 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=23 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=25 / lookahead-slices=4 / scenecut=40 / hist-scenecut=0 / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=3 / limit-modes / me=3 / subme=3 / merange=57 / temporal-mvp / no-frame-dup / no-hme / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / no-sao / no-sao-non-deblock / rd=4 / selective-sao=0 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=1.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=abr / bitrate=2000 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=2 / cplxblur=20.0 / qblur=0.5 / ipratio=1.40 / pbratio=1.30 / aq-mode=3 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=2 / transfer=2 / colormatrix=2 / chromaloc=0 / display-window=0 / cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / hist-threshold=0.01 / no-opt-cu-delta-qp / no-aq-motion / no-hdr10 / no-hdr10-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=0 / analysis-save-reuse-level=0 / analysis-load-reuse-level=0 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=1 / refine-ctu-distortion=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / no-field / qp-adaptation-range=1.00 / no-scenecut-aware-qpconformance-window-offsets / right=0 / bottom=0
Language                                 : English
Default                                  : Yes
Forced                                   : No

Audio #1
ID                                       : 2
Format                                   : AC-3
Format/Info                              : Audio Coding 3
Commercial name                          : Dolby Digital
Codec ID                                 : A_AC3
Duration                                 : 2 min 0 s
Bit rate mode                            : Constant
Bit rate                                 : 256 kb/s
Channel(s)                               : 2 channels
Channel layout                           : L R
Sampling rate                            : 48.0 kHz
Frame rate                               : 31.250 FPS (1536 SPF)
Compression mode                         : Lossy
Delay relative to video                  : -6 ms
Stream size                              : 3.66 MiB (11%)
Language                                 : English
Service kind                             : Complete Main
Default                                  : Yes
Forced                                   : No

Audio #2
ID                                       : 3
Format                                   : AAC LC
Format/Info                              : Advanced Audio Codec Low Complexity
Codec ID                                 : A_AAC-2
Duration                                 : 2 min 0 s
Bit rate                                 : 224 kb/s
Channel(s)                               : 6 channels
Channel layout                           : C L R Ls Rs LFE
Sampling rate                            : 48.0 kHz
Frame rate                               : 46.875 FPS (1024 SPF)
Compression mode                         : Lossy
Delay relative to video                  : -43 ms
Stream size                              : 3.21 MiB (9%)
Language                                 : English
Default                                  : No
Forced                                   : No

Text #1
ID                                       : 4
Format                                   : UTF-8
Codec ID                                 : S_TEXT/UTF8
Codec ID/Info                            : UTF-8 Plain Text
Duration                                 : 54 s 622 ms
Bit rate                                 : 118 b/s
Count of elements                        : 26
Stream size                              : 807 Bytes (0%)
Language                                 : English
Default                                  : Yes
Forced                                   : No

Text #2
ID                                       : 5
Format                                   : UTF-8
Codec ID                                 : S_TEXT/UTF8
Codec ID/Info                            : UTF-8 Plain Text
Duration                                 : 1 min 8 s
Bit rate                                 : 81 b/s
Count of elements                        : 19
Stream size                              : 694 Bytes (0%)
Language                                 : French
Default                                  : No
Forced                                   : No

Text #3
ID                                       : 6
Format                                   : UTF-8
Codec ID                                 : S_TEXT/UTF8
Codec ID/Info                            : UTF-8 Plain Text
Duration                                 : 54 s 137 ms
Bit rate                                 : 124 b/s
Count of elements                        : 26
Stream size                              : 843 Bytes (0%)
Language                                 : Spanish
Default                                  : No
Forced                                   : No

Text #4
ID                                       : 7
Format                                   : UTF-8
Codec ID                                 : S_TEXT/UTF8
Codec ID/Info                            : UTF-8 Plain Text
Duration                                 : 54 s 622 ms
Bit rate                                 : 195 b/s
Count of elements                        : 26
Stream size                              : 1.31 KiB (0%)
Language                                 : Persian
Default                                  : No
Forced                                   : No

Text #5
ID                                       : 8
Format                                   : UTF-8
Codec ID                                 : S_TEXT/UTF8
Codec ID/Info                            : UTF-8 Plain Text
Duration                                 : 54 s 622 ms
Bit rate                                 : 120 b/s
Count of elements                        : 25
Stream size                              : 822 Bytes (0%)
Language                                 : Turkish
Default                                  : No
Forced                                   : No

Here is the AutoProcess.ini regarding subtitles:

[Subtitle]
codec = mov_text
codec-image-based = 
languages = eng, fre, per
default-language = eng
first-stream-of-language = False
encoding = 
burn-subtitles = False
burn-dispositions = 
download-subs = False
download-hearing-impaired-subs = False
download-providers = 
embed-subs = True
embed-image-subs = False
embed-only-internal-subs = False
filename-dispositions = forced
ignore-embedded-subs = False
ignored-dispositions = 
unique-dispositions = False
attachment-codec =

Expected behavior: Subtitles in English, French, and Persian languages are embedded. Subtitles in Turkish and Spanish languages are discarded.

Result: Subtitle in English language is embedded. Subtitles in French, Persian, Turkish, and Spanish languages are discarded.

Any idea?

mdhiggins commented 3 years ago

Please post logs of the conversion for this one

dramirvf commented 3 years ago

Please post logs of the conversion for this one

Here is the log:

2020-11-04 21:47:56 - MANUAL - INFO - Manual processor started.
2020-11-04 21:47:56 - MANUAL - INFO - /usr/local/bin/python3
2020-11-04 21:47:56 - MANUAL - INFO - Loading config file /volume1/Amir Synology/Local SMA/config/autoProcess.ini.
2020-11-04 21:47:59 - MANUAL - INFO - Matched movie title as: The Craft: Legacy 2020-10-28 (TMDB ID: 590995)
2020-11-04 21:47:59 - MANUAL - INFO - Processing The Craft: Legacy
2020-11-04 21:47:59 - MANUAL - INFO - Input Data
2020-11-04 21:47:59 - MANUAL - INFO - {
    "video": {
        "field_order": "unknown",
        "fps": 23.976023976023978,
        "dimensions": "1920x800",
        "index": 0,
        "codec": "hevc",
        "pix_fmt": "yuv420p10le",
        "level": 12.0,
        "profile": "main 10"
    },
    "subtitle": [
        {
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired",
            "language": "eng",
            "index": 3,
            "codec": "subrip"
        },
        {
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired",
            "language": "fra",
            "index": 4,
            "codec": "subrip"
        },
        {
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired",
            "language": "spa",
            "index": 5,
            "codec": "subrip"
        },
        {
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired",
            "language": "fas",
            "index": 6,
            "codec": "subrip"
        },
        {
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired",
            "language": "tur",
            "index": 7,
            "codec": "subrip"
        }
    ],
    "format-fullname": "Matroska / WebM",
    "format": "matroska,webm",
    "attachment": [],
    "audio": [
        {
            "bitrate": 256000,
            "language": "eng",
            "index": 1,
            "codec": "ac3",
            "channels": 2,
            "samplerate": 48000,
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired"
        },
        {
            "language": "eng",
            "index": 2,
            "codec": "aac",
            "channels": 6,
            "samplerate": 48000,
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired"
        }
    ]
}
2020-11-04 21:47:59 - MANUAL - INFO - Reading video stream.
2020-11-04 21:47:59 - MANUAL - INFO - Video codec detected: hevc.
2020-11-04 21:47:59 - MANUAL - INFO - Pix Fmt: yuv420p10le.
2020-11-04 21:47:59 - MANUAL - INFO - Profile: main 10.
2020-11-04 21:47:59 - MANUAL - INFO - Video codec parameters None.
2020-11-04 21:47:59 - MANUAL - INFO - Creating copy video stream from source stream 0.
2020-11-04 21:47:59 - MANUAL - INFO - Reading audio streams.
2020-11-04 21:47:59 - MANUAL - INFO - Audio detected for stream 2 - aac eng 6 channel.
2020-11-04 21:47:59 - MANUAL - INFO - Creating ac3 audio stream source audio stream 2 [universal-audio].
2020-11-04 21:47:59 - MANUAL - INFO - Creating copy audio stream from source stream 2.
2020-11-04 21:47:59 - MANUAL - INFO - Audio detected for stream 1 - ac3 eng 2 channel.
2020-11-04 21:47:59 - MANUAL - INFO - Creating copy audio stream from source stream 1.
2020-11-04 21:47:59 - MANUAL - INFO - Default audio stream set to eng copy 6 channel stream [default-more-channels: True].
2020-11-04 21:47:59 - MANUAL - INFO - Reading subtitle streams.
2020-11-04 21:47:59 - MANUAL - INFO - Text-based subtitle detected for stream 3 - subrip eng.
2020-11-04 21:47:59 - MANUAL - INFO - Creating mov_text subtitle stream from source stream 3.
2020-11-04 21:47:59 - MANUAL - INFO - Text-based subtitle detected for stream 4 - subrip fra.
2020-11-04 21:48:00 - MANUAL - INFO - Text-based subtitle detected for stream 5 - subrip spa.
2020-11-04 21:48:00 - MANUAL - INFO - Text-based subtitle detected for stream 6 - subrip fas.
2020-11-04 21:48:00 - MANUAL - INFO - Text-based subtitle detected for stream 7 - subrip tur.
2020-11-04 21:48:00 - MANUAL - INFO - Scanned for external subtitles and found 0 results in your approved languages.
2020-11-04 21:48:00 - MANUAL - INFO - Tagging copied video stream as hvc1
2020-11-04 21:48:00 - MANUAL - INFO - Output Data
2020-11-04 21:48:00 - MANUAL - INFO - {
    "video": {
        "bitrate": 2340.241,
        "crf": -1,
        "title": "FHD",
        "debug": "video",
        "width": null,
        "maxrate": null,
        "pix_fmt": null,
        "preset": null,
        "field_order": "unknown",
        "map": 0,
        "params": null,
        "filter": null,
        "codec": "copy",
        "level": 0.0,
        "profile": null,
        "bufsize": null
    },
    "subtitle": [
        {
            "language": "eng",
            "title": "",
            "debug": "subtitle.embed-subs",
            "codec": "mov_text",
            "disposition": "-lyrics+default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired",
            "map": 3
        }
    ],
    "format": "mp4",
    "source": [
        "Source - Copy.mkv"
    ],
    "attachment": [],
    "audio": [
        {
            "bitrate": 256,
            "language": "eng",
            "title": "Stereo",
            "filter": null,
            "codec": "ac3",
            "debug": "universal-audio",
            "samplerate": null,
            "channels": 2,
            "map": 2,
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired"
        },
        {
            "bitrate": 1536,
            "language": "eng",
            "title": "5.1 Channel",
            "filter": null,
            "bsf": null,
            "codec": "copy",
            "debug": "audio",
            "samplerate": null,
            "channels": 6,
            "map": 2,
            "disposition": "-lyrics+default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired"
        },
        {
            "bitrate": 256,
            "language": "eng",
            "title": "Stereo",
            "filter": null,
            "bsf": null,
            "codec": "copy",
            "debug": "universal-audio",
            "samplerate": null,
            "channels": 2,
            "map": 1,
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired"
        }
    ]
}
2020-11-04 21:48:00 - MANUAL - INFO - Preopts
2020-11-04 21:48:00 - MANUAL - INFO - [
    "-fix_sub_duration"
]
2020-11-04 21:48:00 - MANUAL - INFO - Postopts
2020-11-04 21:48:00 - MANUAL - INFO - [
    "-threads",
    "0",
    "-metadata:g",
    "encoding_tool=SMA",
    "-tag:v",
    "hvc1"
]
2020-11-04 21:48:00 - MANUAL - INFO - Starting conversion.
2020-11-04 21:48:00 - MANUAL - INFO - FFmpeg command:
2020-11-04 21:48:00 - MANUAL - INFO - ======================
2020-11-04 21:48:00 - MANUAL - INFO - /usr/local/bin/ffmpeg -fix_sub_duration -i Source - Copy.mkv -vcodec copy -map 0:0 -metadata:s:v title=FHD -metadata:s:v handler_name=FHD -c:a:0 ac3 -map 0:2 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0 BPS=256000 -metadata:s:a:0 BPS-eng=256000 -metadata:s:a:0 title=Stereo -metadata:s:a:0 handler_name=Stereo -metadata:s:a:0 language=eng -disposition:a:0 -lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired-captions -c:a:1 copy -map 0:2 -metadata:s:a:1 title=5.1 Channel -metadata:s:a:1 handler_name=5.1 Channel -metadata:s:a:1 language=eng -disposition:a:1 -lyrics+default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired-captions -c:a:2 copy -map 0:1 -metadata:s:a:2 title=Stereo -metadata:s:a:2 handler_name=Stereo -metadata:s:a:2 language=eng -disposition:a:2 -lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired-captions -c:s:0 mov_text -map 0:3 -metadata:s:s:0 title= -metadata:s:s:0 handler_name= -metadata:s:s:0 language=eng -disposition:s:0 -lyrics+default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired-captions -f mp4 -threads 0 -metadata:g encoding_tool=SMA -tag:v hvc1 -y /volume1/video/Test - Test/Tag/Tag 2/2A - 1V - 5T/Source - Copy.mp4
2020-11-04 21:48:00 - MANUAL - INFO - ======================
2020-11-04 21:48:01 - MANUAL - INFO - /volume1/video/Test - Test/Tag/Tag 2/2A - 1V - 5T/Source - Copy.mp4 created.
2020-11-04 21:48:01 - MANUAL - INFO - Tagging file: /volume1/video/Test - Test/Tag/Tag 2/2A - 1V - 5T/Source - Copy.mp4.
2020-11-04 21:48:04 - MANUAL - INFO - Trying to write tags.
2020-11-04 21:48:04 - MANUAL - INFO - Tags written successfully using mutagen.
2020-11-04 21:48:04 - MANUAL - INFO - Relocating MOOV atom to start of file.
mdhiggins commented 3 years ago

There's your issue

    "subtitle": [
        {
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired",
            "language": "eng",
            "index": 3,
            "codec": "subrip"
        },
        {
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired",
            "language": "fra",
            "index": 4,
            "codec": "subrip"
        },
        {
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired",
            "language": "spa",
            "index": 5,
            "codec": "subrip"
        },
        {
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired",
            "language": "fas",
            "index": 6,
            "codec": "subrip"
        },
        {
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired",
            "language": "tur",
            "index": 7,
            "codec": "subrip"
        }

You have used the wrong 3 letter language codes for the 2 languages that aren't working You should use fra not fre and fas not pre or just add all of them to your language list

MediaInfo isnt' super helpful since the script uses FFProbe to pull all the data. You would be better served to check info using that for the future

dramirvf commented 3 years ago

There's your issue

    "subtitle": [
        {
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired",
            "language": "eng",
            "index": 3,
            "codec": "subrip"
        },
        {
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired",
            "language": "fra",
            "index": 4,
            "codec": "subrip"
        },
        {
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired",
            "language": "spa",
            "index": 5,
            "codec": "subrip"
        },
        {
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired",
            "language": "fas",
            "index": 6,
            "codec": "subrip"
        },
        {
            "disposition": "-lyrics-default-original-comment-visual_impaired-forced-karaoke-dub-hearing_impaired",
            "language": "tur",
            "index": 7,
            "codec": "subrip"
        }

You have used the wrong 3 letter language codes for the 2 languages that aren't working You should use fra not fre and fas not pre or just add all of them to your language list

MediaInfo isnt' super helpful since the script uses FFProbe to pull all the data. You would be better served to check info using that for the future

I have written the 3-chacater codes according to "ISO 639-2/B". What you suggested is 3-chacater codes according to "ISO 639-2/T". So all language codes should be based on "ISO 639-2/T"?

mdhiggins commented 3 years ago

Its based on whatever the person who encoded the video you're using as your input decided to use

dramirvf commented 3 years ago

Its based on whatever the person who encoded the video you're using as your input decided to use

So it's looks more logical to keep all variations as below? languages = eng, fre, fra, per, fas

mdhiggins commented 3 years ago

Yeah that should catch all possible scenarios

dramirvf commented 3 years ago

Thanks!