UnicodeDecodeError: 'utf8'

[screwzloose]

I get an interesting error when I try to convert some of my MKVs.

    }, 
    "audio": {
        "0": {
            "channels": 2, 
            "map": 1, 
            "codec": "aac", 
            "bitrate": 256, 
            "language": "eng"
        }, 
        "1": {
            "channels": 6, 
            "map": 1, 
            "codec": "ac3", 
            "bitrate": 1536, 
            "language": "eng"
        }
    }, 
    "video": {
        "map": 0, 
        "codec": "copy", 
        "bitrate": 7526665.0
    }, 
    "format": "mp4"
}
Traceback (most recent call last):
  File "./manual.py", line 299, in <module>
    main()
  File "./manual.py", line 290, in main
    processFile(path, tagdata)
  File "./manual.py", line 188, in processFile
    output = converter.process(inputfile, True)
  File "/mnt/mp4con/mkvtomp4.py", line 120, in process
    outputfile, inputfile = self.convert(inputfile, options, reportProgress)
  File "/mnt/mp4con/mkvtomp4.py", line 422, in convert
    for timecode in conv:
  File "/mnt/mp4con/converter/__init__.py", line 243, in convert
    timeout=timeout):
  File "/mnt/mp4con/converter/ffmpeg.py", line 478, in convert
    ret = ret.decode(console_encoding)
  File "/usr/local/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 8-9: unexpected end of data

media information:

[mdhiggins]

What's the name of the file? Does it have a weird special character?

— Sent from Mailbox

On Wed, Mar 18, 2015 at 5:01 PM, screwzloose notifications@github.com wrote:

I get an interesting error when I try to convert some of my MKVs.

    }, 
    "audio": {
        "0": {
            "channels": 2, 
            "map": 1, 
            "codec": "aac", 
            "bitrate": 256, 
            "language": "eng"
        }, 
        "1": {
            "channels": 6, 
            "map": 1, 
            "codec": "ac3", 
            "bitrate": 1536, 
            "language": "eng"
        }
    }, 
    "video": {
        "map": 0, 
        "codec": "copy", 
        "bitrate": 7526665.0
    }, 
    "format": "mp4"
}
Traceback (most recent call last):
  File "./manual.py", line 299, in <module>
    main()
  File "./manual.py", line 290, in main
    processFile(path, tagdata)
  File "./manual.py", line 188, in processFile
    output = converter.process(inputfile, True)
  File "/mnt/mp4con/mkvtomp4.py", line 120, in process
    outputfile, inputfile = self.convert(inputfile, options, reportProgress)
  File "/mnt/mp4con/mkvtomp4.py", line 422, in convert
    for timecode in conv:
  File "/mnt/mp4con/converter/__init__.py", line 243, in convert
    timeout=timeout):
  File "/mnt/mp4con/converter/ffmpeg.py", line 478, in convert
    ret = ret.decode(console_encoding)
  File "/usr/local/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 8-9: unexpected end of data

media information:

Reply to this email directly or view it on GitHub: https://github.com/mdhiggins/sickbeard_mp4_automator/issues/220

[screwzloose]

"Pirates.Of.The.Caribbean.Dead.Mans.Chest.2006.mkv" w/o quotes, no special characters.

[mdhiggins]

Are you getting this error with lots of files or just this one in particular?

Also what's the full path to the file too? These unicode errors are usually caused by special characters somewhere in the path or the filename.

[screwzloose]

I'm received the error on about 5-6, in the last 3 days I've converted roughly 50 +/- movies (manually). I haven't been able to determine any consistancy.

The path /media/Action/Pirates...

This is within a FreeNAS Jail (running Plex), the absolute path is /mnt/nas/movies/Action/Pirates...

[screwzloose]

Very interesting...

I changed the name of the file from what it was to "potcdeadman.mkv" and it's working... Very odd as most of my movies follow the same naming schema.

[mdhiggins]

I just pushed a commit to try and catch this error and allow things to proceed. It may just create errors further down the line but if you grab the update you can give it a try and see if it works with the previous naming scheme.

[screwzloose]

Tried the new commit to "ffmpeg.py" on a movie it also failed on... New error.

root@Plex:/mnt/mp4con # ./manual.py -i '/media/Action/Blade.mkv' -imdb tt0120611
64
Processing Blade
Video codec detected: h264
Audio stream detected: dca eng [Stream 1]
Creating dual audio channels for iOS compatability for this stream
Attempting to download subtitles, please wait
{
    "subtitle": {}, 
    "audio": {
        "0": {
            "channels": 2, 
            "map": 1, 
            "codec": "aac", 
            "bitrate": 256, 
            "language": "eng"
        }, 
        "1": {
            "channels": 7, 
            "map": 1, 
            "codec": "ac3", 
            "bitrate": 1792, 
            "language": "eng"
        }
    }, 
    "video": {
        "map": 0, 
        "codec": "copy", 
        "bitrate": 10181572.0
    }, 
    "format": "mp4"
}
Traceback (most recent call last):
  File "./manual.py", line 299, in <module>
    main()
  File "./manual.py", line 290, in main
    processFile(path, tagdata)
  File "./manual.py", line 188, in processFile
    output = converter.process(inputfile, True)
  File "/mnt/mp4con/mkvtomp4.py", line 120, in process
    outputfile, inputfile = self.convert(inputfile, options, reportProgress)
  File "/mnt/mp4con/mkvtomp4.py", line 422, in convert
    for timecode in conv:
  File "/mnt/mp4con/converter/__init__.py", line 243, in convert
    timeout=timeout):
  File "/mnt/mp4con/converter/ffmpeg.py", line 524, in convert
    line, pid=p.pid)
converter.ffmpeg.FFMpegConvertError: <FFMpegConvertError error="Error while opening encoder for output stream #0:2 - maybe incorrect parameters such as bit_rate, rate, width or height", pid=34281, cmd="/usr/local/bin/ffmpeg -i /media/Action/Blade.mkv -vcodec copy -map 0:0 -c:a:0 aac -map 0:1 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0 language=eng -strict experimental -c:a:1 ac3 -map 0:1 -ac:a:1 7 -metadata:s:a:1 language=eng -f mp4 -threads auto -y /media/Action/Blade.mp4">

[mdhiggins]

If you take that FFMPEG path that the error puts out and run that manually does the conversion work?

/usr/local/bin/ffmpeg -i /media/Action/Blade.mkv -vcodec copy -map 0:0 -c:a:0 aac -map 0:1 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0 language=eng -strict experimental -c:a:1 ac3 -map 0:1 -ac:a:1 7 -metadata:s:a:1 language=eng -f mp4 -threads auto -y /media/Action/Blade.mp4

Also I'm noticing that the bitrate for the 7 channel audio stream isn't getting passed. Lemme see if I can correct that real quick

[mdhiggins]

Just pushed another commit that should catch what was causing your bitrate to not be passed, that might be what was happening. Give that a try with blade again.

If you get a TypeError its cause I screwed up. Already fixed it with another commit but just in case you grabbed the update before I got a chance to correct that mistake, giving you a heads up.

[screwzloose]

Error I'm getting with the new commits.

root@Plex:/mnt/mp4con # ./manual.py -i '/media/Action/Blade.mkv' -imdb tt0120611                                                                64
Processing Blade
Video codec detected: h264
Audio stream detected: dca eng [Stream 1]
Creating dual audio channels for iOS compatability for this stream
Attempting to download subtitles, please wait
{
    "subtitle": {}, 
    "audio": {
        "0": {
            "channels": 2, 
            "map": 1, 
            "codec": "aac", 
            "bitrate": 256, 
            "language": "eng"
        }, 
        "1": {
            "channels": 7, 
            "map": 1, 
            "codec": "ac3", 
            "bitrate": 1792, 
            "language": "eng"
        }
    }, 
    "video": {
        "map": 0, 
        "codec": "copy", 
        "bitrate": 10181572.0
    }, 
    "format": "mp4"
}
Traceback (most recent call last):
  File "./manual.py", line 299, in <module>
    main()
  File "./manual.py", line 290, in main
    processFile(path, tagdata)
  File "./manual.py", line 188, in processFile
    output = converter.process(inputfile, True)
  File "/mnt/mp4con/mkvtomp4.py", line 120, in process
    outputfile, inputfile = self.convert(inputfile, options, reportProgress)
  File "/mnt/mp4con/mkvtomp4.py", line 422, in convert
    for timecode in conv:
  File "/mnt/mp4con/converter/__init__.py", line 243, in convert
    timeout=timeout):
  File "/mnt/mp4con/converter/ffmpeg.py", line 524, in convert
    line, pid=p.pid)
converter.ffmpeg.FFMpegConvertError: <FFMpegConvertError error="Error while opening encoder for output stream #0:2 - maybe incorrect parameters such as bit_rate, rate, width or height", pid=34795, cmd="/usr/local/bin/ffmpeg -i /media/Action/Blade.mkv -vcodec copy -map 0:0 -c:a:0 aac -map 0:1 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0 language=eng -strict experimental -c:a:1 ac3 -map 0:1 -ac:a:1 7 -b:a:1 1536k -metadata:s:a:1 language=eng -f mp4 -threads auto -y /media/Action/Blade.mp4">
root@Plex:/mnt/mp4con # 

when I run the FFMPEG manually, mentioned above, I get the following.

root@Plex:/mnt/mp4con # /usr/local/bin/ffmpeg -i /media/Action/Blade.mkv -vcodec copy -map 0:0 -c:a:0 aac -map 0:1 -ac:a:0 2 -b:a:0 256k -metadata:s:a:0 language=eng -strict experimental -c:a:1 ac3 -map 0:1 -ac:a:1 7 -metadata:s:a:1 language=eng -f mp4 -threads auto -y /media/Action/Blade.mp4
ffmpeg version 2.3.6 Copyright (c) 2000-2014 the FFmpeg developers
  built on Mar 10 2015 23:51:17 with gcc 4.2.1 (GCC) 20070831 patched [FreeBSD]
  configuration: --disable-libaacplus --disable-indev=alsa --disable-outdev=alsa --disable-libopencore-amrnb --disable-libopencore-amrwb --disable-libass --disable-libcdio --disable-libcelt --disable-libfaac --disable-libfdk-aac --enable-ffserver --enable-fontconfig --enable-libfreetype --enable-frei0r --enable-gnutls --disable-libgsm --enable-iconv --disable-indev=jack --disable-libmp3lame --disable-libbluray --disable-libv4l2 --disable-indev=v4l2 --disable-outdev=v4l2 --disable-libmodplug --disable-openal --disable-indev=openal --enable-libopencv --disable-libopenjpeg --disable-openssl --disable-libopus --disable-libpulse --disable-indev=pulse --disable-outdev=pulse --disable-librtmp --enable-libschroedinger --disable-libspeex --enable-libtheora --disable-vaapi --disable-vdpau --enable-libvorbis --disable-libvo-aacenc --disable-libvo-amrwbenc --enable-libvpx --enable-libx264 --disable-libx265 --enable-libxvid --prefix=/usr/local --mandir=/usr/local/man --datadir=/usr/local/share/ffmpeg --enable-shared --enable-gpl --enable-postproc --enable-avfilter --enable-avresample --enable-pthreads --enable-memalign-hack --disable-libstagefright-h264 --disable-libutvideo --disable-libsoxr --cc=cc --extra-cflags='-msse -I/usr/local/include/vorbis -I/usr/local/include' --extra-ldflags='-L/usr/local/lib ' --extra-libs=-pthread --disable-debug --disable-ffplay --disable-outdev=sdl
  libavutil      52. 92.100 / 52. 92.100
  libavcodec     55. 69.100 / 55. 69.100
  libavformat    55. 48.100 / 55. 48.100
  libavdevice    55. 13.102 / 55. 13.102
  libavfilter     4. 11.100 /  4. 11.100
  libavresample   1.  3.  0 /  1.  3.  0
  libswscale      2.  6.100 /  2.  6.100
  libswresample   0. 19.100 /  0. 19.100
  libpostproc    52.  3.100 / 52.  3.100
[dca @ 0x808820b20] Number of channels changed in DCA decoder (6 -> 7)
Input #0, matroska,webm, from '/media/Action/Blade.mkv':
  Metadata:
    encoder         : libebml v1.3.0 + libmatroska v1.4.0
    creation_time   : 2013-04-15 12:56:37
  Duration: 02:00:16.63, start: 0.000000, bitrate: 10181 kb/s
    Stream #0:0(eng): Video: h264 (High), yuv420p, 1920x800, SAR 1:1 DAR 12:5, 23.98 fps, 23.98 tbr, 1k tbn, 47.95 tbc (default)
    Stream #0:1(eng): Audio: dts (DTS-ES), 48000 Hz, 6.1, fltp, 1536 kb/s
    Metadata:
      title           : US DTS
[AVFilterGraph @ 0x80883e0e0] [Eval @ 0x7fffffff9530] Undefined constant or missing '(' in 'auto'
[AVFilterGraph @ 0x80883e0e0] Unable to parse option value "auto"
[AVFilterGraph @ 0x808840360] [Eval @ 0x7fffffff9530] Undefined constant or missing '(' in 'auto'
[AVFilterGraph @ 0x808840360] Unable to parse option value "auto"
[ac3 @ 0x808824220] Specified channel layout '6.1' is not supported
Output #0, mp4, to '/media/Action/Blade.mp4':
  Metadata:
    encoder         : libebml v1.3.0 + libmatroska v1.4.0
    Stream #0:0(eng): Video: h264, yuv420p, 1920x800 [SAR 1:1 DAR 12:5], q=2-31, 23.98 fps, 1k tbn, 1k tbc (default)
    Stream #0:1(eng): Audio: aac, 48000 Hz, stereo, fltp, 256 kb/s
    Metadata:
      title           : US DTS
      encoder         : Lavc55.69.100 aac
    Stream #0:2(eng): Audio: ac3, 0 channels, 128 kb/s
    Metadata:
      title           : US DTS
      encoder         : Lavc55.69.100 ac3
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (dts (dca) -> aac (native))
  Stream #0:1 -> #0:2 (dts (dca) -> ac3 (native))
Error while opening encoder for output stream #0:2 - maybe incorrect parameters such as bit_rate, rate, width or height

Any chance you got a PayPal donate, I'd like to toss you a few bones, as a thanks for your commitment to the script.

[mdhiggins]

I see what's happening. Here's the error [ac3 @ 0x808824220] Specified channel layout '6.1' is not supported

I just pushed a new commit that will specifically override any channels > 6 for AC3. I think that will fix your error.

and sure! donations are always appreciated. My paypal email is tehpsyc@gmail.com. Thanks :)

[screwzloose]

It's working for the Blade.mkv. I'm going to hit a few others that failed and report back. Well done Sir.

[mdhiggins]

Nice, keep me posted.

[screwzloose]

Processed over 20+ movies w/o issues, thank you sir!

[mdhiggins]

Excellent, gonna close this for now. Open up a new issue if you have any further problems.

and thanks so much for the donation. Much appreciated.