search

lisamelton / video_transcoding

Tools to transcode, inspect and convert videos.
MIT License
2.39k stars 160 forks source link

transcoding failed on a 640x480 video #195

Closed ProxyCell closed 6 years ago

ProxyCell commented 6 years ago

I'm not able to even transcode one video that I have in a large group. It does transcode in the HandBrake GUI, CLI and also with ffmpeg but not under the very straight forward usage of transcode-video.

transcode-video input.avi

EDIT: I just wanted to add that its a little strange that its failing on this one when it works on everything else. The only significant difference is that this video is 640x480 and the rest are 1280x720. It is also encoded with xvid and mp3 - everything else is h264 and aac.

I have a long log file here: [21:51:41] hb_init: starting libhb thread [21:51:41] thread 700009465000 started ("libhb") HandBrake 1.0.7 (2017100100) - Darwin x86_64 - https://handbrake.fr 8 CPUs detected Opening input.avi... [21:51:41] CPU: Intel(R) Core(TM) i7-3820QM CPU @ 2.70GHz [21:51:41] - Intel microarchitecture Ivy Bridge [21:51:41] - logical processor count: 8 [21:51:41] hb_scan: path=input.avi, title_index=1 udfread ERROR: ECMA 167 Volume Recognition failed disc.c:274: failed opening UDF image input.avi disc.c:352: error opening file BDMV/index.bdmv disc.c:352: error opening file BDMV/BACKUP/index.bdmv [21:51:41] bd: not a bd - trying as a stream/file instead libdvdnav: Using dvdnav version 5.0.1 libdvdread: Encrypted DVD support unavailable. libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed libdvdread: Can't open file VIDEO_TS.IFO. libdvdnav: vm: failed to read VIDEO_TS.IFO [21:51:41] dvd: not a dvd - trying as a stream/file instead Input #0, avi, from 'input.avi': Metadata: encoder : VirtualDubMod 1.4.13 Duration: 00:23:15.02, start: 0.000000, bitrate: 4140 kb/s Stream #0:0: Video: mpeg4 (Simple Profile) [XVID / 0x44495658] yuv420p, 640x480 [PAR 1:1 DAR 4:3] 29.97 fps, 29.97 tbn Stream #0:1: Audio: mp3 [U[0][0][0] / 0x0055] 44100 Hz, 2 channels, s16p, 128 kb/s [21:51:41] scan: decoding previews for title 1 [21:51:42] scan: audio 0x1: mp3, rate=44100Hz, bitrate=128000 Unknown (MP3) (2.0 ch) [21:51:42] scan: 10 previews, 640x480, 29.970 fps, autocrop = 0/0/0/0, aspect 4:3, PAR 1:1 [21:51:42] libhb: scan thread found 1 valid title(s)

lisamelton commented 6 years ago

@ProxyCell Thanks for including your .log file output! I reformatted it to be a little more readable.

OK, when you say "a large group," do you mean several other .avi files in the same folder or on the same drive? Do any of your other files work with transcode-video?

The only time you should ever get this generic sounding "transcoding failed" error message is for a serious system call failure. So, is your current working directory read-only or perhaps full?

ProxyCell commented 6 years ago

It is part of a group of 95 videos - 94 transcoded perfectly fine but I noticed this one didn't come through and ran it again and found this error

The rest are all h264/aac/720 while this one is xvid/mp3/480

The drive and folder permissions should be fine. The others were transcoded in the same directory and I have gone on to transcode other files since that batch.

Update: I'm able to re-run the other 94 files it seems. I just restarted them again and it seems to be fine in the same folder. I let the first three complete and then stopped the rest.

lisamelton commented 6 years ago

@ProxyCell Thanks for the quick response!

OK, the error indicates a catastrophic failure propagated by HandBrakeCLI. In fact, the failure appears to be so bad that HandBrakeCLI couldn't even print an error message to the console. :)

So, I need you to invoke HandBrakeCLIdirectly with exactly the same parameters that transcode-video would be passing to it. Fortunately, that's incredibly easy with the --dry-run option! Just do this at your command line in the same working directory:

transcode-video --dry-run input.avi

...and then copy the output of that command and paste it back into your console.

That way we can take transcode-video out of the equation and see what happens.

ProxyCell commented 6 years ago

Alright! Thats an awesome feature by the way.

HandBrakeCLI --input=input.avi --output=input.mkv --markers --encoder=x264 --crop=0:0:0:0 --auto-anamorphic --deinterlace --rate=30 --pfr --encoder-preset=medium --encoder-profile=high --encoder-level=3.0 --quality=1 --audio=1 --aencoder=ca_aac --encopts=vbv-maxrate=1500:vbv-bufsize=3000:crf-max=25:qpmax=34

[22:20:55] hb_init: starting libhb thread
[22:20:55] thread 700004f42000 started ("libhb")
HandBrake 1.0.7 (2017100100) - Darwin x86_64 - https://handbrake.fr
8 CPUs detected
Opening input.avi...
[22:20:55] CPU: Intel(R) Core(TM) i7-3820QM CPU @ 2.70GHz
[22:20:55]  - Intel microarchitecture Ivy Bridge
[22:20:55]  - logical processor count: 8
[22:20:55] hb_scan: path=input.avi, title_index=1
udfread ERROR: ECMA 167 Volume Recognition failed
disc.c:274: failed opening UDF image input.avi
disc.c:352: error opening file BDMV/index.bdmv
disc.c:352: error opening file BDMV/BACKUP/index.bdmv
[22:20:55] bd: not a bd - trying as a stream/file instead
libdvdnav: Using dvdnav version 5.0.1
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.IFO failed
libdvdread:DVDOpenFileUDF:UDFFindFile /VIDEO_TS/VIDEO_TS.BUP failed
libdvdread: Can't open file VIDEO_TS.IFO.
libdvdnav: vm: failed to read VIDEO_TS.IFO
[22:20:55] dvd: not a dvd - trying as a stream/file instead
Input #0, avi, from 'input.avi':
  Metadata:
    encoder         : VirtualDubMod 1.4.13
  Duration: 00:23:15.02, start: 0.000000, bitrate: 4140 kb/s
    Stream #0:0: Video: mpeg4 (Simple Profile) [XVID / 0x44495658]
      yuv420p, 640x480 [PAR 1:1 DAR 4:3]
      29.97 fps, 29.97 tbn
    Stream #0:1: Audio: mp3 [U[0][0][0] / 0x0055]
      44100 Hz, 2 channels, s16p, 128 kb/s
[22:20:55] scan: decoding previews for title 1
[22:20:55] scan: audio 0x1: mp3, rate=44100Hz, bitrate=128000 Unknown (MP3) (2.0 ch)
[22:20:55] scan: 10 previews, 640x480, 29.970 fps, autocrop = 0/0/0/0, aspect 4:3, PAR 1:1
[22:20:55] libhb: scan thread found 1 valid title(s)
+ Using preset: CLI Default
+ title 1:
  + stream: input.avi
  + duration: 00:23:15
  + size: 640x480, pixel aspect: 1/1, display aspect: 1.33, 29.970 fps
  + autocrop: 0/0/0/0
  + support opencl: no
  + chapters:
    + 1: cells 0->0, 0 blocks, duration 00:23:15
  + audio tracks:
    + 1, Unknown (MP3) (2.0 ch) (iso639-2: und)
  + subtitle tracks:
[22:20:55] 1 job(s) to process
[22:20:55] json job:
{
    "Audio": {
        "AudioList": [
            {
                "Encoder": 16384,
                "Track": 0
            }
        ],
        "CopyMask": [
            "copy:aac",
            "copy:ac3",
            "copy:eac3",
            "copy:dtshd",
            "copy:dts",
            "copy:mp3",
            "copy:truehd",
            "copy:flac"
        ],
        "FallbackEncoder": "ca_aac"
    },
    "Destination": {
        "ChapterList": [
            {
                "Name": ""
            }
        ],
        "ChapterMarkers": false,
        "File": "input.mkv",
        "Mp4Options": {
            "IpodAtom": false,
            "Mp4Optimize": false
        },
        "Mux": "mkv"
    },
    "Filters": {
        "FilterList": [
            {
                "ID": 5,
                "Settings": {
                    "mode": "3"
                }
            },
            {
                "ID": 6,
                "Settings": {
                    "mode": 2,
                    "rate": "27000000/900000"
                }
            },
            {
                "ID": 11,
                "Settings": {
                    "crop-bottom": 0,
                    "crop-left": 0,
                    "crop-right": 0,
                    "crop-top": 0,
                    "height": 480,
                    "width": 640
                }
            }
        ]
    },
    "Metadata": {},
    "PAR": {
        "Den": 1,
        "Num": 1
    },
    "SequenceID": 0,
    "Source": {
        "Angle": 0,
        "Path": "input.avi",
        "Range": {
            "End": 1,
            "Start": 1,
            "Type": "chapter"
        },
        "Title": 1
    },
    "Subtitle": {
        "Search": {
            "Burn": true,
            "Default": false,
            "Enable": false,
            "Forced": false
        },
        "SubtitleList": []
    },
    "Video": {
        "ColorMatrixCode": 0,
        "Encoder": "x264",
        "Level": "3.0",
        "OpenCL": false,
        "Options": "vbv-maxrate=1500:vbv-bufsize=3000:crf-max=25:qpmax=34",
        "Preset": "medium",
        "Profile": "high",
        "QSV": {
            "AsyncDepth": 4,
            "Decode": false
        },
        "Quality": 1.0,
        "Tune": "",
        "Turbo": false,
        "TwoPass": false
    }
}
[22:20:55] starting job
[22:20:55] work: mixdown not specified, track 1 setting mixdown Stereo
[22:20:55] work: bitrate not specified, track 1 setting bitrate 160 Kbps
[22:20:55] job configuration:
[22:20:55]  * source
[22:20:55]    + input.avi
[22:20:55]    + title 1, chapter(s) 1 to 1
[22:20:55]    + container: avi
[22:20:55]    + data rate: 4140 kbps
[22:20:55]  * destination
[22:20:55]    + input.mkv
[22:20:55]    + container: Matroska (libavformat)
[22:20:55]  * video track
[22:20:55]    + decoder: mpeg4
[22:20:55]    + filters
[22:20:55]      + avfilter (yadif='mode=0')
[22:20:55]        + yadif: mode=0
[22:20:55]      + Framerate Shaper (mode=2:rate=27000000/900000)
[22:20:55]        + frame rate: 29.970 fps -> peak rate limited to 30.000 fps
[22:20:55]      + Crop and Scale (width=640:height=480:crop-top=0:crop-bottom=0:crop-left=0:crop-right=0)
[22:20:55]        + source: 640 * 480, crop (0/0/0/0): 640 * 480, scale: 640 * 480
[22:20:55]    + Output geometry
[22:20:55]      + storage dimensions: 640 x 480
[22:20:55]      + pixel aspect ratio: 1 : 1
[22:20:55]      + display dimensions: 640 x 480
[22:20:55]    + encoder: H.264 (libx264)
[22:20:55]      + preset:  medium
[22:20:55]      + options: vbv-maxrate=1500:vbv-bufsize=3000:crf-max=25:qpmax=34
[22:20:55]      + profile: high
[22:20:55]      + level:   3.0
[22:20:55]      + quality: 1.00 (RF)
[22:20:55]  * audio track 1
[22:20:55]    + decoder: Unknown (MP3) (2.0 ch) (track 1, id 0x1)
[22:20:55]      + bitrate: 128 kbps, samplerate: 44100 Hz
[22:20:55]    + mixdown: Stereo
[22:20:55]    + encoder: AAC (Apple AudioToolbox)
[22:20:55]      + bitrate: 160 kbps, samplerate: 44100 Hz
[22:20:55] sync: expecting 41809 video frames
[22:20:56] encx264: min-keyint: 30, keyint: 300
[22:20:56] encx264: encoding at constant RF 1.000000
[22:20:56] encx264: unparsed options: crf-max=25:vbv-maxrate=1500:level=3.0:vbv-bufsize=3000:qpmax=34
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x264 [info]: profile High, level 3.0
[22:20:56] sync: first pts video is 0
[22:20:56] sync: Chapter 1 at frame 1 time 0
[22:20:56] sync: first pts audio 0x1 is 0
Segmentation fault: 11
lisamelton commented 6 years ago

@ProxyCell This case is exactly why I included that feature. :)

OK, now we know it's HandBrakeCLI dying a horrible death and not transcode-video. However, that doesn't excuse my code yet.

Here's what you need to try next, but be warned that it's very tedious. :) This is your command, right?

HandBrakeCLI --input=input.avi --output=input.mkv --markers --encoder=x264 --crop=0:0:0:0 --auto-anamorphic --deinterlace --rate=30 --pfr --encoder-preset=medium --encoder-profile=high --encoder-level=3.0 --quality=1 --audio=1 --aencoder=ca_aac --encopts=vbv-maxrate=1500:vbv-bufsize=3000:crf-max=25:qpmax=34

So just start removing options and arguments, one at time, and see if it suddenly starts working. Then we'll know if transcode-video was passing it something it couldn't handle.

BTW, if this input file is small enough and there aren't any copyright restrictions, is it possible to upload this somewhere that I can inspect it and try it myself?

ProxyCell commented 6 years ago

Alright - I ran through them and its --deinterlace

I can upload this easily, its just a 2008 episode of a Japanese variety show. Its 722mb but the quality is horrendous. So I am surprised that it detects interlaced frames as I'm sure its already deinteraced. Let me know if you would like this thrown up somewhere for you to download.

ProxyCell commented 6 years ago

Additional: I'm going to run it with this new line now. I just reduced it to the plain command arguments earlier to try and eliminate any possible issues. Again, the --deinterlace option is on and removing it lets it pass.

transcode-video --dry-run --mp4 --veryquick --burn-srt "subtitle.srt" input.avi

HandBrakeCLI --input=input.avi --output=input.mp4 --markers --encoder=x264 --crop=0:0:0:0 --auto-anamorphic --deinterlace --rate=30 --pfr --encoder-preset=medium --encoder-profile=high --encoder-level=3.0 --quality=1 --audio=1 --aencoder=ca_aac --srt-file=subtitle.srt --srt-burn --encopts=vbv-maxrate=1500:vbv-bufsize=3000:crf-max=25:qpmax=34:partitions=none:ref=1:bframes=1:rc-lookahead=30:me=dia

lisamelton commented 6 years ago

@ProxyCell It's not HandBrakeCLI that thinks your video is interlaced. My code automatically applies that filter if the input frame rate is 29.97 and the video is not MPEG-2 format.

Try this to work around the problem: 

transcode-video -H _deinterlace input.avi

Be sure to use the _ (underscore) there. That should remove the --deinterlace option from the command line sent to HandBrakeCLI.

Thanks for offering to upload the file but I know exactly what's wrong now.

What I don't know is why HandBrakeCLI is dying on that. That's clearly a bug.

ProxyCell commented 6 years ago

Thanks a lot! I never came across the issue when using HandBrake-GUI or CLI, ffmpeg and such because I never ran de-interlacing on it.

Lets close this ticket off as its a bug in the upstream.

Now let me heap some praise on you man: I've spent MONTHS encoding my blu rays, DVDs, etc to h265 and then kept having to deal with two issues: speed - it took MONTHS, right? and the other was compatibility with some devices/software.

I came across a blog article which dealt with the benchmarks and quality comparison of different presets on h264 for HandBrake and your video_transcoding. It praised your suite of tools so I decided to give h264 a try again and ran my own quality comparison tests. I made 10 different encodes of a 6 minute video of a live concert and passed them around to friends to rank them in terms of quality.

The --quick and --veryquick were the top two.

It may have been because the source was a 1080i Blu Ray of a live concert in a dark venue but still, I was convinced to switch over. Your suite has probably saved me MONTHS in just the past few days already as I used to encode at the VERYSLOW preset for handbrake/ffmpeg.

While I have the storage space and bandwidth to run the encodes on a CDN my company possesses, these servers aren't really meant to encode and are terrifyingly slow. They are all built just to serve audio/video-streams as quickly as possible. Just one of the benefits of working at this company :)

lisamelton commented 6 years ago

@ProxyCell You are very welcome and thanks so much for using my tools! I'm glad they're working so well for you. This is exactly why I do it. :)

Just so you know, I use --veryquickmyself to transcode my entire video library. And that library is huge. Why waste time, right? :)

In fact, my command line typically looks like this:

transcode-video --abr --veryquick --audio-width main=surround "/path/to/Movie.mkv"

And they look great! Honestly it's very difficult to tell any of them from transcodings done using the default medium preset or slow. Even on my 80-inch TV. I have to get a foot away from my 27-inch iMac screen to tell the difference. And that's only for a few small scenes in a few videos.

I'm glad you like --veryquick too!

ProxyCell commented 6 years ago

Yeah. I decided its best to spend my time WATCHING rather than ENCODING!

Any chance that "--abr --veryquick --target small" can be combined?

lisamelton commented 6 years ago

@ProxyCell I wouldn't combine those since they're all very different.