mpv hangs with mvtools vapoursynth script

[4re]

Linux x86_64 nvidia-drivers-343.22 mpv 0.6.0 Vapoursynth Core r24 mvtools from git (https://github.com/dubhater/vapoursynth-mvtools) Script:

import vapoursynth as vs
core = vs.get_core()

src = video_in
clip = core.std.Trim(src, first=0, length=500000)

if clip.format.bits_per_sample != 8:
    clip = core.fmtc.bitdepth(clip, bits=8)

super = core.mv.Super(clip, pel=2)
bv    = core.mv.Analyse(super, isb=True, overlap=4, search=4)
fv    = core.mv.Analyse(super, isb=False, overlap=4, search=4)
clip  = core.mv.FlowInter(clip, super, bv, fv)
clip  = core.std.Interleave([src, clip])

clip.set_output()

I don't have problems using other vapoursynth filters, just this one. Also no meaningful output on the terminal, it just hangs.

[ghost]

It hangs because mvtools requests a frame at initialization. Since vf_vapoursynth can deliver frames only after initialization has finished, it's not possible to fix this on mpv's side.

[ghost]

<nodame> wm4: Well I guess you're out of luck then.

Too bad!

[4re]

It was a beautiful dream while it lasted :(

[ghost]

You still could do something convoluted, like piping vspipe output to mpv.

[ghost]

Some relevant info from the author of the filters:

`

Filters that take a super clip will want some information that Super passes along in frame properties. Filters that take motion vectors will want some information that Analyse passes along in the frames. It's how it's always been done™. If arbitrary properties could be attached to nodes, it would not be necessary to request frames during filter creation. `

[bjin]

For reference, I met similar issue before (#1026).

[bjin]

@wm4 I think there might be workarounds for this issue as well. From the description from the filter author, Analyse requires some frame properties from output of Super and hence makes a frame request. But for Super, it doesn't necessarily need frame from upstream of mpv filter chain. I think providing dummy video frames for whole vapoursynth filter during filter initialization will probably solve this issue smoothly.

EDIT: I think I made a mistake. The first few output frames of the filter will probably become dummy as well then, but at least the remaining part of video should be fine. It's ugly but better than deadlock or filter failure from my opinion.

[ghost]

If we feed it dummy frames, we'll also get dummy output.

[bjin]

Yes, I realized that later. But it's only for first few frames.

[ghost]

It'll also happen after seeks.

[bjin]

It's still a workaround, but this could be avoided with customized vapoursynth script, user can append few dummy frame in front, and seek to real part later.

[ghost]

There. Doesn't work overly well, but I suppose it's better than failing completely.

[ghost]

So does this work for you?

[bjin]

It turns out that my laptop is too slow to run this script smoothly(roughly 10fps for a 576p footage), the result looks as expected though(have blank dummy frame in the first ones). I also tried to reduce the blank frames by appending extra frame on the front, but it didn't work.

Anyway I don't use mvtools for realtime playback myself. Although I'm interested in @4re's feedback as well.

[4re]

Sorry for the late reply.

I gave a shot at your recommendations and it works! somewhat. Sometimes it stutters but the cpu load is not full. I'm not 100% sure this happens because some threads top at some given time as this only happens with 720p or more. It also is quite sensitive and makes mpv crash if I add more filters or try to seek, but not always.

Other than that it's great to have it running, thank you guys!

[ghost]

Sometimes it stutters but the cpu load is not full.

Uh, not sure. Maybe try playing with the threading suboptions of vf_vapoursynth.

It also is quite sensitive and makes mpv crash if I add more filters or try to seek, but not always.

This shouldn't happen. Could be a bug in mpv or the filters you use; hard to tell.

[ghedo]

I get the "crashes on seek" too. However it doesn't happen on every seek. The error is:

mpv: ../src/core/vsapi.cpp:64: const uint8_t* getReadPtr(const VSFrameRef*, int): Assertion `frame' failed.

i.e. it crashes from inside vapoursynth. May just be a vs bug...

[ghost]

Actually I'm getting it quite often. MVClipBalls::Update(VSFrameRef const*) crashes in libmvtools.so, probably because mpv returns errors on frame requests when it wants to terminate the filter (like on seek). I suppose it's a bug in the mvtools filter.

[dubhater]

If your source filter calls setFilterError, you're supposed to get a null VSFrameRef* at the other end. I don't think filters are supposed to deal with null VSFrameRef*.

[ghost]

If your source filter calls setFilterError, you're supposed to get a null VSFrameRef* at the other end. I don't think filters are supposed to deal with null VSFrameRef*.

Not sure if I follow. What else should it return if not a null VSFrameRef*? Though I could make it return dummy frames on EOF. It will probably be a bit slower, but trip up fewer broken filters.

[dubhater]

Your source filter, video_in, does need to call setFilterError and return NULL. This NULL is only supposed to appear at the end of the chain, where you call getFrame/getFrameAsync. It should not appear in any of the filters in-between, because the core is not supposed to execute any more filters after setFilterError was called. At least this is how I hope VapourSynth is meant to work.

[ghost]

Your source filter, video_in, does need to call setFilterError and return NULL.

It does do that. In the beginning it returned just NULL, but that was fixed some months ago.

Anyway, the vapoursynth dev could reproduce this problem outside of mpv. It seems there is a race in error handling. Using vs.get_core(threads=1) in the script seems to make the crash go away.

[4re]

The crashes are gone after https://github.com/vapoursynth/vapoursynth/commit/8ef3b6ced9f11d7fecd265186416b22e3a64bb0d

[ghost]

I'm still getting deadlocks with that, and I think a follow-up fix is in work.

[ghost]

I think this should be fully fixed now in vapoursynth git master?

[4re]

For me everything is fixed, thank you.

[Sperg]

Could you PLEASE post the finished script. I've been trying to get frame interpolation happening on GNU/Linux for a week now, this looks to be the beacon of hope I've been searching for. I was just about ready to learn the ffmpeg C library. And this will do frame interpolation right? Right guys?

[Sperg]

Just incase that was the finished product I tried your script and got this

┌─[mugi]─[/mpv/mpv-master] └──╼ ./build/mpv --vf=vapoursynth=/script.vpy --vo=opengl ~/fuck.mp4 Playing: /root/fuck.mp4 [stream] Video (+) --vid=1 () (h264) [stream] Audio (+) --aid=1 --alang=und () (aac) File tags: major_brand: mp42 minor_version: 0 compatible_brands: isommp42 creation_time: 2014-01-15 20:42:30 Opening video filter: [vapoursynth file=/script.vpy] AO: [pulse] 44100Hz stereo 2ch float [vapoursynth] Could not get script output node. Video filter chain: [vd] 1280x720 yuv420p Autoselect/Autoselect CL=1 [in] 1280x720 yuv420p BT.709 (HD)/TV CL=1 [vapoursynth] 1280x720 yuv420p BT.709 (HD)/TV CL=1 <--- [out] ??? Cannot initialize video filters. Could not initialize video chain. Video: no video

fuck.mp4 is that "What The F@#% AMV" by the way, don't judge me.

I have an r9 270x if that makes any difference.

[Sperg]

Ok I fixed it never mind.

[rr-]

Using OP's script, for some videos I get

[vapoursynth] Frame requested during init! This is unsupported.
[vapoursynth] Returning black dummy frame with 0 duration.
[vapoursynth] Frame requested during init! This is unsupported.
[vapoursynth] Returning black dummy frame with 0 duration.
zsh: floating point exception (core dumped)  mpv 

and with this slightly changed script:

import vapoursynth as vs

core = vs.get_core()
clip = video_in

if not (clip.width > 1920 or clip.height > 1080 or container_fps > 59):
    clip = core.std.AssumeFPS(clip, fpsnum = int(container_fps * 1e8), fpsden = int(1e8))
    sup = core.mv.Super(clip, pel=2, hpad=0, vpad=0)
    bvec = core.mv.Analyse(sup, blksize=16, isb=True, chroma=True, search=3, searchparam=1)
    fvec = core.mv.Analyse(sup, blksize=16, isb=False, chroma=True, search=3, searchparam=1)
    clip = core.mv.BlockFPS(clip, sup, bvec, fvec, num=60000, den=1001, mode=3, thscd2=12)

clip.set_output()

I get a bit different error message:

[vapoursynth] Frame requested during init! This is unsupported.
[vapoursynth] Returning black dummy frame with 0 duration.
[vapoursynth] Frame requested during init! This is unsupported.
[vapoursynth] Returning black dummy frame with 0 duration.
Attempted to read from a map with error set: Filter creation aborted, zero (unknown) and negative length clips not allowed
zsh: abort (core dumped)  mpv 

[ghost]

Useless without backtrace...

[rr-]

Using OP's script:

Program received signal SIGFPE, Arithmetic exception.
0x00007ffff666e822 in ?? () from /usr/lib/libvapoursynth.so
(gdb) bt
#0  0x00007ffff666e822 in ?? () from /usr/lib/libvapoursynth.so
#1  0x00007ffff668fbdd in VSPlugin::invoke(std::string const&, VSMap const&) () from /usr/lib/libvapoursynth.so
#2  0x00007ffff667e680 in ?? () from /usr/lib/libvapoursynth.so
#3  0x00007fffc50f8220 in ?? () from /usr/lib/python3.4/site-packages/vapoursynth.so
#4  0x00007fffe9e2b528 in PyObject_Call (func=func@entry=0x7fffc47bd5a0, arg=arg@entry=0x7fffc47bec50, kw=kw@entry=0x0) at Objects/abstract.c:2040
#5  0x00007fffe9edd4ea in do_call (nk=<optimized out>, na=<optimized out>, pp_stack=0x7fffffffd7d0, func=<optimized out>) at Python/ceval.c:4466
#6  call_function (oparg=<optimized out>, pp_stack=0x7fffffffd7d0) at Python/ceval.c:4264
#7  PyEval_EvalFrameEx (f=0x7fffc5320240, throwflag=<optimized out>) at Python/ceval.c:2838
#8  0x00007fffe9ee0947 in PyEval_EvalCodeEx (_co=0x0, globals=0x0, locals=0x0, args=0x0, argcount=2, argcount@entry=0, kws=0x7fffffffce00, kws@entry=0x0, kwcount=-993783760, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0)
    at Python/ceval.c:3588
#9  0x00007fffe9ee09eb in PyEval_EvalCode (co=<optimized out>, globals=<optimized out>, locals=<optimized out>) at Python/ceval.c:775
#10 0x00007fffc50eedb7 in vpy_evaluateScript () from /usr/lib/python3.4/site-packages/vapoursynth.so
#11 0x00007fffc50efcb5 in vpy_evaluateFile () from /usr/lib/python3.4/site-packages/vapoursynth.so
#12 0x00000000004b4fd9 in drv_vss_load (vf=0x875960, vars=<optimized out>) at ../video/filter/vf_vapoursynth.c:793
#13 0x00000000004b5720 in reinit_vs (vf=vf@entry=0x875960) at ../video/filter/vf_vapoursynth.c:622
#14 0x00000000004b61e3 in config (vf=0x875960, width=1280, height=720, d_width=<optimized out>, d_height=<optimized out>, flags=0, fmt=<optimized out>) at ../video/filter/vf_vapoursynth.c:665
#15 0x00000000004afb63 in vf_reconfig_wrapper (p=<optimized out>, vf=<optimized out>) at ../video/filter/vf.c:591
#16 vf_reconfig (c=0x874d30, params=params@entry=0x7fffffffdf20, override_params=override_params@entry=0x7fffffffde80) at ../video/filter/vf.c:631
#17 0x00000000004ac596 in video_reconfig_filters (d_video=d_video@entry=0x8749b0, params=params@entry=0x7fffffffdf20) at ../video/decode/dec_video.c:411
#18 0x000000000047f997 in filter_reconfig (mpctx=mpctx@entry=0x7f4360, probe_only=probe_only@entry=false) at ../player/video.c:108
#19 0x000000000047fc6b in video_filter (mpctx=mpctx@entry=0x7f4360, eof=eof@entry=false) at ../player/video.c:454
#20 0x000000000047fd02 in video_decode_and_filter (mpctx=mpctx@entry=0x7f4360) at ../player/video.c:495
#21 0x0000000000480ed7 in video_output_image (endpts=-9.2233720368547758e+18, mpctx=0x7f4360) at ../player/video.c:662
#22 write_video (mpctx=mpctx@entry=0x7f4360, endpts=endpts@entry=-9.2233720368547758e+18) at ../player/video.c:800
#23 0x000000000047ca3e in run_playloop (mpctx=mpctx@entry=0x7f4360) at ../player/playloop.c:1028
#24 0x0000000000474d32 in play_current_file (mpctx=0x7f4360) at ../player/loadfile.c:1240
#25 mp_play_files (mpctx=mpctx@entry=0x7f4360) Program received signal SIGABRT, Aborted.

Using my script:

Program received signal SIGABRT, Aborted.
0x00007fffef89e528 in raise () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007fffef89e528 in raise () from /usr/lib/libc.so.6
#1  0x00007fffef89f93a in abort () from /usr/lib/libc.so.6
#2  0x00007ffff6693d99 in vsLog(char const*, long, VSMessageType, char const*, ...) () from /usr/lib/libvapoursynth.so
#3  0x00007ffff667f1d6 in ?? () from /usr/lib/libvapoursynth.so
#4  0x00007fffad2cb039 in ?? () from /usr/lib/vapoursynth/libmvtools.so
#5  0x00007ffff668fbdd in VSPlugin::invoke(std::string const&, VSMap const&) () from /usr/lib/libvapoursynth.so
#6  0x00007ffff667e680 in ?? () from /usr/lib/libvapoursynth.so
#7  0x00007fffc50f8220 in ?? () from /usr/lib/python3.4/site-packages/vapoursynth.so
#8  0x00007fffe9e2b528 in PyObject_Call (func=func@entry=0x7fffc47bd5a0, arg=arg@entry=0x7fffc47bf138, kw=kw@entry=0x7fffc531a448) at Objects/abstract.c:2040
#9  0x00007fffe9edd4ea in do_call (nk=<optimized out>, na=<optimized out>, pp_stack=0x7fffffffd7d0, func=<optimized out>) at Python/ceval.c:4466
#10 call_function (oparg=<optimized out>, pp_stack=0x7fffffffd7d0) at Python/ceval.c:4264
#11 PyEval_EvalFrameEx (f=0x7fffc5399848, throwflag=<optimized out>) at Python/ceval.c:2838
#12 0x00007fffe9ee0947 in PyEval_EvalCodeEx (_co=0x4181, globals=0x4181, locals=0x6, args=0x0, argcount=-135411584, argcount@entry=0, kws=0x4d, kws@entry=0x0, kwcount=-993783760, defs=0x0, defcount=0, kwdefs=0x0, closure=0x0)
    at Python/ceval.c:3588
#13 0x00007fffe9ee09eb in PyEval_EvalCode (co=<optimized out>, globals=<optimized out>, locals=<optimized out>) at Python/ceval.c:775
#14 0x00007fffc50eedb7 in vpy_evaluateScript () from /usr/lib/python3.4/site-packages/vapoursynth.so
#15 0x00007fffc50efcb5 in vpy_evaluateFile () from /usr/lib/python3.4/site-packages/vapoursynth.so
#16 0x00000000004b4fd9 in drv_vss_load (vf=0x875960, vars=<optimized out>) at ../video/filter/vf_vapoursynth.c:793
#17 0x00000000004b5720 in reinit_vs (vf=vf@entry=0x875960) at ../video/filter/vf_vapoursynth.c:622
#18 0x00000000004b61e3 in config (vf=0x875960, width=1280, height=720, d_width=<optimized out>, d_height=<optimized out>, flags=0, fmt=<optimized out>) at ../video/filter/vf_vapoursynth.c:665
#19 0x00000000004afb63 in vf_reconfig_wrapper (p=<optimized out>, vf=<optimized out>) at ../video/filter/vf.c:591
#20 vf_reconfig (c=0x874d30, params=params@entry=0x7fffffffdf20, override_params=override_params@entry=0x7fffffffde80) at ../video/filter/vf.c:631
#21 0x00000000004ac596 in video_reconfig_filters (d_video=d_video@entry=0x8749b0, params=params@entry=0x7fffffffdf20) at ../video/decode/dec_video.c:411
#22 0x000000000047f997 in filter_reconfig (mpctx=mpctx@entry=0x7f4360, probe_only=probe_only@entry=false) at ../player/video.c:108
#23 0x000000000047fc6b in video_filter (mpctx=mpctx@entry=0x7f4360, eof=eof@entry=false) at ../player/video.c:454
#24 0x000000000047fd02 in video_decode_and_filter (mpctx=mpctx@entry=0x7f4360) at ../player/video.c:495
#25 0x0000000000480ed7 in video_output_image (endpts=-9.2233720368547758e+18, mpctx=0x7f4360) at ../player/video.c:662
#26 write_video (mpctx=mpctx@entry=0x7f4360, endpts=endpts@entry=-9.2233720368547758e+18) at ../player/video.c:800
#27 0x000000000047ca3e in run_playloop (mpctx=mpctx@entry=0x7f4360) at ../player/playloop.c:1028
#28 0x0000000000474d32 in play_current_file (mpctx=0x7f4360) at ../player/loadfile.c:1240
#29 mp_play_files (mpctx=mpctx@entry=0x7f4360) at ../player/loadfile.c:1392
#30 0x0000000000475cfc in mpv_main (argc=<optimized out>, argv=0x7fffffffe358) at ../player/main.c:533
#31 0x00007fffef88b790 in __libc_start_main () from /usr/lib/libc.so.6
#32 0x00000000004152e9 in _start ()

Let me know if I should compile vapoursynth with debug symbols.

[ghost]

Maybe it's some sort of overflow in the clip length calculation. But I think it must be internal to vapoursynth then.

[ghost]

You can try reducing this number: int enough_for_everyone = INT_MAX / 16; in vf_vapoursynth.c.

[ghost]

Also don't hijack issues with different issues.

[kabalage]

num=60, den=1 instead of 60000, 1001 works until this is fixed