FallingSnow / h265ize

A node utility utilizing ffmpeg to encode videos with the hevc codec.
MIT License
527 stars 81 forks source link

loudnorm aka --normalize-level 4 fails with "Unhandled rejection Error: ENOENT: no such file or directory, unlink 'FILE.mkv'" #105

Open djbclark opened 7 years ago

djbclark commented 7 years ago

Using Bleeding edge 0.5.0 2017-08-23 4:49pm EDT installed via npm install FallingSnow/h265ize --global

Using sample media files from https://github.com/mediaelement/mediaelement-files

$ h265ize --normalize-level 4 --debug big_buck_bunny.mp4

The h265ize.log.txt doesn't look to have anything useful. The console output error is:

[h265ize]: Flushing log to disk...
Unhandled rejection Error: ENOENT: no such file or directory, unlink '/Users/djbclark/github/mediaelement-files/h265/big_buck_bunny.mkv'
    at Object.fs.unlinkSync (fs.js:1067:18)
    at /usr/local/lib/node_modules/h265ize/lib/classes/video.js:1258:20
    at arrayEach (/usr/local/lib/node_modules/h265ize/node_modules/lodash/lodash.js:537:11)
    at Function.forEach (/usr/local/lib/node_modules/h265ize/node_modules/lodash/lodash.js:9359:14)
    at /usr/local/lib/node_modules/h265ize/lib/classes/video.js:1257:15
    at Video.cleanUp (/usr/local/lib/node_modules/h265ize/lib/classes/video.js:1256:16)
    at Video.stop (/usr/local/lib/node_modules/h265ize/lib/classes/video.js:1251:21)
    at Encoder.removeVideo (/usr/local/lib/node_modules/h265ize/lib/classes/encoder.js:176:19)
    at EventEmitter.<anonymous> (/usr/local/lib/node_modules/h265ize/lib/classes/encoder.js:147:23)
    at emitOne (events.js:115:13)
    at EventEmitter.emit (events.js:210:7)
    at /usr/local/lib/node_modules/h265ize/lib/classes/video.js:1181:26
From previous event:
    at Video.start (/usr/local/lib/node_modules/h265ize/lib/classes/video.js:1173:34)
    at Encoder.loop (/usr/local/lib/node_modules/h265ize/lib/classes/encoder.js:151:15)
    at Encoder.start (/usr/local/lib/node_modules/h265ize/lib/classes/encoder.js:73:18)
    at /usr/local/lib/node_modules/h265ize/h265ize:255:25
    at <anonymous>
    at runMicrotasksCallback (internal/process/next_tick.js:121:5)
    at _combinedTickCallback (internal/process/next_tick.js:131:7)
    at process._tickCallback (internal/process/next_tick.js:180:9)

[h265ize]: Process ended.

However I think that is also not useful, just complaining that a file it was expecting didn't exist because of ffmpeg failing. However if you manually run the ffmpeg command from the log

[h265ize]: [verbose] Running stage: Encode
[h265ize]: [debug] Running Query: ffmpeg -n 10 /usr/local/bin/ffmpeg -i big_buck_bunny.mp4 -y -acodec copy -filter:a loudnorm=I=-16:TP=-2.0:LRA=11:measured_I=-26.13:measured_LRA=18.00:measured_TP=-4.10:measured_thresh=-37.64:offset=1.56:linear=true:print_format=summary -vcodec libx265 -c:s copy -c:d copy -pix_fmt yuv420p -map 0:1 -map 0:0 -metadata:s:a:0 title="English AAC LC (Stereo)" -crf 19 -preset fast /Users/djbclark/github/mediaelement-files/h265/big_buck_bunny.mkv
[h265ize]: Video failed processing at Wednesday, August 23rd 2017, 4:32:36 PM

There seems to be one or two problems; if the logging is to be believed it looks like maybe there is a typo and it is running "ffmpeg" instead of "nice" with the error you'd expect from this mistake:

$ ffmpeg -n 10 /usr/local/bin/ffmpeg -i big_buck_bunny.mp4 -y -acodec copy -filter:a loudnorm=I=-16:TP=-2.0:LRA=11:measured_I=-26.13:measured_LRA=18.00:measured_TP=-4.10:measured_thresh=-37.64:offset=1.56:linear=true:print_format=summary -vcodec libx265 -c:s copy -c:d copy -pix_fmt yuv420p -map 0:1 -map 0:0 -metadata:s:a:0 title="English AAC LC (Stereo)" -crf 19 -preset fast /Users/djbclark/github/mediaelement-files/h265/big_buck_bunny.mkv
[...]
[NULL @ 0x7fd5b1836000] Unable to find a suitable output format for '10'
10: Invalid argument

However stripping off the "ffmpeg -n 10" still leaves an error:

$ /usr/local/bin/ffmpeg -i big_buck_bunny.mp4 -y -acodec copy -filter:a loudnorm=I=-16:TP=-2.0:LRA=11:measured_I=-26.13:measured_LRA=18.00:measured_TP=-4.10:measured_thresh=-37.64:offset=1.56:linear=true:print_format=summary -vcodec libx265 -c:s copy -c:d copy -pix_fmt yuv420p -map 0:1 -map 0:0 -metadata:s:a:0 title="English AAC LC (Stereo)" -crf 19 -preset fast /Users/djbclark/github/mediaelement-files/h265/big_buck_bunny.mkv && echo YES || echo NO
[...]
Filtergraph 'loudnorm=I=-16:TP=-2.0:LRA=11:measured_I=-26.13:measured_LRA=18.00:measured_TP=-4.10:measured_thresh=-37.64:offset=1.56:linear=true:print_format=summary' was defined for audio output stream 0:1 but codec copy was selected.
Filtering and streamcopy cannot be used together.
NO

I've tried with both HEAD ffmpeg compiled from source and a ffmpeg static snapshot build, both separately and from within h265ize using: export FFMPEG_PATH="/Applications/ffmpeg-static" with the same results.

$ /usr/local/bin/ffmpeg -version
ffmpeg version git-2017-08-23-eca2a49 Copyright (c) 2000-2017 the FFmpeg developers
built with Apple LLVM version 8.1.0 (clang-802.0.42)
configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD-eca2a49 --enable-shared --enable-pthreads --enable-gpl --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-libass --enable-libfdk-aac --enable-libfontconfig --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-librtmp --enable-libsoxr --enable-libtesseract --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-opencl --enable-videotoolbox --enable-openssl --disable-lzma --enable-nonfree --enable-vda
libavutil      55. 74.100 / 55. 74.100
libavcodec     57.103.100 / 57.103.100
libavformat    57. 76.100 / 57. 76.100
libavdevice    57.  7.101 / 57.  7.101
libavfilter     6.100.100 /  6.100.100
libavresample   3.  6.  0 /  3.  6.  0
libswscale      4.  7.102 /  4.  7.102
libswresample   2.  8.100 /  2.  8.100
libpostproc    54.  6.100 / 54.  6.100
$ /Applications/ffmpeg-static -version
ffmpeg version N-87010-g3c99523-tessus Copyright (c) 2000-2017 the FFmpeg developers
built with Apple LLVM version 8.0.0 (clang-800.0.42.1)
configuration: --cc=/usr/bin/clang --prefix=/opt/ffmpeg --extra-version=tessus --enable-avisynth --enable-fontconfig --enable-gpl --enable-libass --enable-libbluray --enable-libfreetype --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopus --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libtheora --enable-libvidstab --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libx264 --enable-libx265 --enable-libxavs --enable-libxvid --enable-libzmq --enable-libzvbi --enable-version3 --disable-ffplay --disable-indev=qtkit
libavutil      55. 74.100 / 55. 74.100
libavcodec     57.102.100 / 57.102.100
libavformat    57. 76.100 / 57. 76.100
libavdevice    57.  7.100 / 57.  7.100
libavfilter     6. 99.100 /  6. 99.100
libswscale      4.  7.102 /  4.  7.102
libswresample   2.  8.100 /  2.  8.100
libpostproc    54.  6.100 / 54.  6.100

Also h265ize --version seems to be broken, gives:

Unabled to retrieve version: Error: [local-git-sync] no git repository found
    at _getGitDirectory (/usr/local/lib/node_modules/h265ize/node_modules/local-git-sync/index.js:46:11)
    at _getGitDirectory (/usr/local/lib/node_modules/h265ize/node_modules/local-git-sync/index.js:62:10)
    at _getGitDirectory (/usr/local/lib/node_modules/h265ize/node_modules/local-git-sync/index.js:62:10)
    at _getGitDirectory (/usr/local/lib/node_modules/h265ize/node_modules/local-git-sync/index.js:62:10)
    at _getGitDirectory (/usr/local/lib/node_modules/h265ize/node_modules/local-git-sync/index.js:62:10)
    at _getGitDirectory (/usr/local/lib/node_modules/h265ize/node_modules/local-git-sync/index.js:62:10)
    at Object.branch (/usr/local/lib/node_modules/h265ize/node_modules/local-git-sync/index.js:66:16)
    at /usr/local/lib/node_modules/h265ize/h265ize:100:38
    at Promise.cancellationExecute [as _execute] (/usr/local/lib/node_modules/h265ize/node_modules/bluebird/js/release/debuggability.js:321:9)
    at Promise._resolveFromExecutor (/usr/local/lib/node_modules/h265ize/node_modules/bluebird/js/release/promise.js:483:18)
    at new Promise (/usr/local/lib/node_modules/h265ize/node_modules/bluebird/js/release/promise.js:79:10)
    at runCli (/usr/local/lib/node_modules/h265ize/h265ize:62:5)
    at Object.<anonymous> (/usr/local/lib/node_modules/h265ize/h265ize:278:5)
    at Module._compile (module.js:573:30)
    at Object.Module._extensions..js (module.js:584:10)
    at Module.load (module.js:507:32)
    at tryModuleLoad (module.js:470:12)
    at Function.Module._load (module.js:462:3)
    at Function.Module.runMain (module.js:609:10)
    at startup (bootstrap_node.js:158:16)
    at bootstrap_node.js:578:3

But looking at it manually shows the latest 0.5.0 git seems to be installed:

$ cat /usr/local/lib/node_modules/h265ize/package.json | grep version
  "version": "0.5.0"
djbclark commented 7 years ago

These links look possibly apropos:

Dual-pass mode requires two calls to the loudnorm filter, the first of which is for measurement. This command will print loudness stats to stderr as JSON, and create no output file. Of course, dual-pass normalization can be easily scripted.

ffmpeg -i in.wav -af loudnorm=I=-16:TP=-1.5:LRA=11:print_format=json -f null -

{
        "input_i" : "-27.61",
        "input_tp" : "-4.47",
        "input_lra" : "18.06",
        "input_thresh" : "-39.20",
        "output_i" : "-16.58",
        "output_tp" : "-1.50",
        "output_lra" : "14.78",
        "output_thresh" : "-27.71",
        "normalization_type" : "dynamic",
        "target_offset" : "0.58"
}

We can then use these stats to call FFmpeg for a second time. Supplying the filter with these stats allow it to make more intelligent normalization decisions, as well as normalize linearly if possible. This second pass creates the loudness normalized output file, and prints a human-readable stats summary to stderr.

ffmpeg -i in.wav -af loudnorm=I=-16:TP=-1.5:LRA=11:measured_I=-27.61:measured_LRA=18.06:measured_TP=-4.47:measured_thresh=-39.20:offset=0.58:linear=true:print_format=summary -ar 48k out.wav

Input Integrated:    -27.5 LUFS
Input True Peak:      -4.5 dBTP
Input LRA:            18.1 LU
Input Threshold:     -39.2 LUFS

Output Integrated:   -16.0 LUFS
Output True Peak:     -1.5 dBTP
Output LRA:           14.6 LU
Output Threshold:    -27.2 LUFS

Normalization Type:   Dynamic
Target Offset:        +0.0 LU
FallingSnow commented 7 years ago

Filtering and streamcopy cannot be used together.

By default h265ize copys audio unmodified. By using loudnorm audio normalization you are telling ffmpeg to normalize (modify) but copy (don't modify) the audio. This is a bug with h265ize, h265ize should either present a warning about this or default to he audio. A quick fix is to use the --he-audio flag. Otherwise I suggest running the ffmpeg command with your own audio settings (codec, bitrate, channels, etc.).

djbclark commented 7 years ago

Hmm, so as you say --he-audio does work. I was actually using --downmix-he-audio originally but left it out for simplicity in the bug report; and that still doesn't work, same error as reported originally.

However --he-audio --downmix-he-audio does work. So there is also a bug with --downmix-he-audio since the doc says that option should enable he-audio:

  --downmix-he-audio  Downmix he-audio opus to Dolby Pro Logic II at 40
                      kbps/channel. Enables he-audio. [boolean] [default: false]

For safety I think I'm going to go with

--normalize-level 4 --he-audio --force-he-audio --downmix-he-audio

:-)

Thanks for the great software!

djbclark commented 7 years ago

Hmm actually looking at the doc again I'm not sure what exactly the difference is between --he-audio and --downmix-he-audio ... I think I may have read the doc too quickly and assumed downmix meant to stereo, and would take less room, but I guess they are both 40 kbs/sec, with downmix actually being from stereo to surround sound?

Dolby Pro Logic II transforms traditional stereo audio into 5.1-channel surround sound for a seamless listening experience full of presence and depth.

So --downmix-he-audio seems like maybe a bit confusing, in that it is not a downmix from the --he-audio option, but instead a possible downmix from some original 5.1 audio codec source, but actually aimed at 5.1 speaker setups? Or am I still confused?

Or tl;dr: If I am h265izing mono and stereo college lectures, which of the audio options should I use?

FallingSnow commented 7 years ago

However --he-audio --downmix-he-audio does work. So there is also a bug with --downmix-he-audio since the doc says that option should enable he-audio:

Your right, that is another bug.

For safety I think I'm going to go with --normalize-level 4 --he-audio --force-he-audio --downmix-he-audio

--force-he-audio doesn't do what I think you think it does. It encodes lossless streams, such as those encoded with flac, with the opus codec (something someone wouldn't want to do by accident (lose a lot of quality)).

Hmm actually looking at the doc again I'm not sure what exactly the difference is between --he-audio and --downmix-he-audio

--downmix-he-audio downmixes any audio stream with more than 3 channels to 2 channel (ex. 7.1 surround -> Stereo). Videos that already have stereo or even mono audio will not be affected.

So --downmix-he-audio seems like maybe a bit confusing, in that it is not a downmix from the --he-audio option, but instead a possible downmix from some original 5.1 audio codec source, but actually aimed at 5.1 speaker setups? Or am I still confused?

DPLII is a stereo audio format that DPLII capable surround systems can efficiently upmix to 5.1. So If you start with a video that has 5.1 channel audio and want to reencode it to a file as small as possible you can use this DPLII "compression" scheme to bring that 5.1 down to stereo which can be brought back close to the original 5.1 during playback. Hopefully that makes sense.

Or tl;dr: If I am h265izing mono and stereo college lectures, which of the audio options should I use?

Just --he-audio.

Thanks for the great software!

You're welcome!

Btw, just wanted to say thanks for the detailed bug report, usually don't see these.