search

FFMS / ffms2

An FFmpeg based source library and Avisynth/VapourSynth plugin for easy frame accurate access
Other
588 stars 104 forks source link

interoperability with avxsynth broken #184

Closed kaefert closed 9 years ago

kaefert commented 9 years ago

The commit 7847d0de1b7706e24e82bdaedf1b6fe09334294d in this repository causes this issue with avxsynth for me: With the latest version from repository any call to ffms2 like FFVideoSource() makes avxsynth hang indefinitely

For a more detailed explanation see: https://github.com/avxsynth/avxsynth/issues/110

The last commit that works for me is this one 08f94eea89fa3de4f68184f8c7eebe32a21130a8 git checkout 08f94eea89fa3de4f68184f8c7eebe32a21130a8

dwbuiten commented 9 years ago

I can't imagine what AVXSynth is doing to have this commit cause it - all avformat_network_init() does is call any required Winsock (on Windows only) and OpenSSL/GnuTLS *_init() funcs which are global. On Linux, without TLS support it's literally a no-op.

It baffles me what AvxSynth could be doing that causes a hang here.

Can you provide a GDB or MSVC (or WinDBG if you hate life) backtrace to where it is hanging?

kaefert commented 9 years ago

Ähm. could you give me a pointer as to how to generate what you want? I'm runing Linux Mint 17.1 (~ Ubuntu 14.04 LTS)

dwbuiten commented 9 years ago

Build AvxSynth, FFmpeg/Libav, and FFMS2 with debug enabled, and then run whatever thing you use AvxSynth with under gdb:

gdb --args ./someavxsynthapp myffmsscript.avs

Type r in the prompt.

When it hangs, press CTRL+C, and use:

set logging on
bt full
q

It should produce gdb.txt and dump a load to your screen. The info in gdb.txt is what is useful here.

Google should have lots of info about debugging freezes/deadlocks too.

kaefert commented 9 years ago

will this do?

kaefert@ultrablech ~/src/ffms2 $ gdb --args ~/src/mpv/build/mpv /home/kaefert/Videos/test.avs
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/kaefert/src/mpv/build/mpv...done.
(gdb) r
Starting program: /home/kaefert/src/mpv/build/mpv /home/kaefert/Videos/test.avs
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffec924700 (LWP 13565)]
Playing: /home/kaefert/Videos/test.avs
[New Thread 0x7fffec123700 (LWP 13566)]
[Thread 0x7fffec123700 (LWP 13566) exited]
[New Thread 0x7fffec123700 (LWP 13567)]

Program received signal SIGINT, Interrupt.
0x00007ffff326f771 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb) set logging on
Copying output to gdb.txt.
(gdb) bt full
#0  0x00007ffff326f771 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
No symbol table info available.
#1  0x0000000000440d29 in mp_input_wait (ictx=0x753ec0, seconds=100) at ../input/input.c:854
        ts = {tv_sec = 1419535773, tv_nsec = 821439754}
#2  0x000000000046ac0a in mp_wait_events (sleeptime=<optimized out>, mpctx=0x746050) at ../player/playloop.c:59
No locals.
#3  mp_idle (mpctx=0x746050) at ../player/playloop.c:1020
No locals.
#4  0x00000000004673b2 in mpctx_run_non_blocking (mpctx=mpctx@entry=0x746050, thread_fn=thread_fn@entry=0x462ec0 <open_demux_thread>, thread_arg=thread_arg@entry=0x7fffffffdef0) at ../player/misc.c:296
        args = {mpctx = 0x746050, thread_fn = 0x462ec0 <open_demux_thread>, thread_arg = 0x7fffffffdef0, mutex = {__data = {__lock = 0, __count = 0, __owner = 0, __nusers = 0, __kind = 0, __spins = 0, __elision = 0, __list = {
                __prev = 0x0, __next = 0x0}}, __size = '\000' <repeats 39 times>, __align = 0}, done = false}
        success = false
        thread = 140737154004736
#5  0x0000000000464e96 in open_demux_async (stream=<optimized out>, mpctx=0x746050) at ../player/loadfile.c:889
        args = {stream = 0x7fffeb720050, global = 0x77cb70, demux = 0x0}
#6  play_current_file (mpctx=0x746050) at ../player/loadfile.c:1004
        opts = 0x7492a0
        stream_flags = <optimized out>
        tmp = 0x772050
        startpos = <optimized out>
        nothing_played = <optimized out>
        playback_start = -1e+100
        end_event = {reason = -344850352, error = 32767}
#7  mp_play_files (mpctx=mpctx@entry=0x746050) at ../player/loadfile.c:1320
        new_entry = <optimized out>
#8  0x000000000046685c in mpv_main (argc=<optimized out>, argv=<optimized out>) at ../player/main.c:542
        mpctx = 0x746050
        opts = 0x7492a0
        verbose_env = <optimized out>
        r = <optimized out>
#9  0x00007ffff2ebcec5 in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6
No symbol table info available.
#10 0x000000000040edbe in _start ()
No symbol table info available.
(gdb) q
A debugging session is active.

    Inferior 1 [process 13561] will be killed.

Quit anyway? (y or n) y

UPDATE1: or if this is not enough, how do I "debug enabled" my builds of FFmpeg, ffms2 & avxsynth? I'm currently using this guide to build them: http://forum.doom9.org/showpost.php?p=1643184&postcount=193

ghost commented 9 years ago

Can you run in gdb instead of bt full the following command:

thread apply all backtrace
dubhater commented 9 years ago

Just out of curiosity, what do you do with Avxsynth that you can't do with VapourSynth?

kaefert commented 9 years ago

Hi there. I have never heard of VapourSynth until you mentioned it here. I'm gonna take a look at it in the next few days. I use Avxsynth as the most awesome non-linear video editing tool that very much fits the way I like to talk to computers ;)

UPDATE1: I've taken a quick look at VapourSynth. The scripting language is Python.. Which on the one hand is of course more powerful since its a general purpose language, but on the other hand its much more complex and requires much more writing than avs scripts.

UPDATE2: VapourSynth is incapable of replacing Avxsynth for me because of its poor audio support. For me a clip [by default] consists of a video and an audio stream, and I want both to be carried through from the beginning to the end of my video editing process. Of course if you only have one input clip where you want to filter the picture in some way, you can easily handle audio manually, but when you have tens or hundreds of input clips that you want to cut together (read: playing hobby director) then that is near impossible with VapourSynth.

kaefert@ultrablech ~ $ gdb --args ~/src/mpv/build/mpv /home/kaefert/Videos/test.avs
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.2) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /home/kaefert/src/mpv/build/mpv...done.
(gdb) r
Starting program: /home/kaefert/src/mpv/build/mpv /home/kaefert/Videos/test.avs
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffec924700 (LWP 11584)]
Playing: /home/kaefert/Videos/test.avs
[New Thread 0x7fffec123700 (LWP 11585)]
[Thread 0x7fffec123700 (LWP 11585) exited]
[New Thread 0x7fffec123700 (LWP 11586)]

Program received signal SIGINT, Interrupt.
0x00007ffff326f771 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb) set logging on
Copying output to gdb.txt.
(gdb) thread apply all backtrace

Thread 4 (Thread 0x7fffec123700 (LWP 11586)):
#0  0x00007ffff326ff2c in __lll_lock_wait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x00007ffff326b657 in _L_lock_909 () from /lib/x86_64-linux-gnu/libpthread.so.0
#2  0x00007ffff326b480 in pthread_mutex_lock () from /lib/x86_64-linux-gnu/libpthread.so.0
#3  0x00007ffff6419ea3 in ?? () from /usr/local/lib/libavcodec.so.56
#4  0x00007ffff6420630 in avpriv_lock_avformat () from /usr/local/lib/libavcodec.so.56
#5  0x00007ffff5c3da39 in ?? () from /usr/local/lib/libavformat.so.56
#6  0x00007ffff5ca3b8f in avformat_network_init () from /usr/local/lib/libavformat.so.56
#7  0x00007fffe822c38e in FFMS_Init () from /usr/local/lib/libffms2.so.3
#8  0x00007fffe84610f7 in ?? () from /usr/local/lib/avxsynth//libavxffms2.so
#9  0x00007fffeb466ac9 in avxsynth::ScriptEnvironment::Invoke(char const*, avxsynth::AVSValue, char const**) () from /usr/local/lib/libavxsynth.so
#10 0x00007fffeb46fcaa in avxsynth::ExpFunctionCall::Call(avxsynth::IScriptEnvironment*) () from /usr/local/lib/libavxsynth.so
#11 0x00007fffeb4701fb in avxsynth::ExpFunctionCall::Evaluate(avxsynth::IScriptEnvironment*) () from /usr/local/lib/libavxsynth.so
#12 0x00007fffeb46ef3b in avxsynth::ExpAssignment::Evaluate(avxsynth::IScriptEnvironment*) () from /usr/local/lib/libavxsynth.so
#13 0x00007fffeb46f318 in avxsynth::ExpExceptionTranslator::ChainEval(avxsynth::AVSValue&, avxsynth::IScriptEnvironment*) () from /usr/local/lib/libavxsynth.so
#14 0x00007fffeb46f42c in avxsynth::ExpExceptionTranslator::Evaluate(avxsynth::IScriptEnvironment*) () from /usr/local/lib/libavxsynth.so
#15 0x00007fffeb46f8d6 in avxsynth::ExpLine::Evaluate(avxsynth::IScriptEnvironment*) () from /usr/local/lib/libavxsynth.so
#16 0x00007fffeb46d6b2 in avxsynth::ExpSequence::Evaluate(avxsynth::IScriptEnvironment*) () from /usr/local/lib/libavxsynth.so
#17 0x00007fffeb46d6b2 in avxsynth::ExpSequence::Evaluate(avxsynth::IScriptEnvironment*) () from /usr/local/lib/libavxsynth.so
#18 0x00007fffeb474ce0 in avxsynth::Eval(avxsynth::AVSValue, void*, avxsynth::IScriptEnvironment*) () from /usr/local/lib/libavxsynth.so
#19 0x00007fffeb466ac9 in avxsynth::ScriptEnvironment::Invoke(char const*, avxsynth::AVSValue, char const**) () from /usr/local/lib/libavxsynth.so
#20 0x00007fffeb475219 in avxsynth::Import(avxsynth::AVSValue, void*, avxsynth::IScriptEnvironment*) () from /usr/local/lib/libavxsynth.so
#21 0x00007fffeb466ac9 in avxsynth::ScriptEnvironment::Invoke(char const*, avxsynth::AVSValue, char const**) () from /usr/local/lib/libavxsynth.so
#22 0x00007fffeb4620d0 in avs_invoke () from /usr/local/lib/libavxsynth.so
#23 0x00007ffff5b8c4bd in ?? () from /usr/local/lib/libavformat.so.56
#24 0x00007ffff5ca2df6 in avformat_open_input () from /usr/local/lib/libavformat.so.56
#25 0x0000000000431538 in demux_open_lavf (demuxer=0x7fffe40011b0, check=<optimized out>) at ../demux/demux_lavf.c:757
#26 0x000000000042dd7e in open_given_type (check=DEMUX_CHECK_UNSAFE, params=0x0, stream=0x7fffeb720050, desc=0x4d12c0 <demuxer_desc_lavf>, log=0x7fffe4000900, global=0x77cb80) at ../demux/demux.c:894
#27 demux_open (stream=0x7fffeb720050, force_format=<optimized out>, params=params@entry=0x0, global=0x77cb80) at ../demux/demux.c:957
#28 0x0000000000462edc in open_demux_thread (pctx=0x7fffffffdf20) at ../player/loadfile.c:882
#29 0x000000000046698e in thread_wrapper (pctx=0x7fffffffde80) at ../player/misc.c:276
#30 0x00007ffff3269182 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#31 0x00007ffff2f95efd in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 2 (Thread 0x7fffec924700 (LWP 11584)):
#0  0x00007ffff2f88bad in poll () from /lib/x86_64-linux-gnu/libc.so.6
#1  0x00000000004c2db3 in poll (__timeout=1000, __nfds=2, __fds=0x7fffec923e90) at /usr/include/x86_64-linux-gnu/bits/poll2.h:41
#2  terminal_thread (ptr=<optimized out>) at ../osdep/terminal-unix.c:395
#3  0x00007ffff3269182 in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#4  0x00007ffff2f95efd in clone () from /lib/x86_64-linux-gnu/libc.so.6

Thread 1 (Thread 0x7ffff7fb7780 (LWP 11580)):
#0  0x00007ffff326f771 in sem_timedwait () from /lib/x86_64-linux-gnu/libpthread.so.0
#1  0x0000000000440d29 in mp_input_wait (ictx=0x753ec0, seconds=100) at ../input/input.c:854
#2  0x000000000046ac0a in mp_wait_events (sleeptime=<optimized out>, mpctx=0x746050) at ../player/playloop.c:59
#3  mp_idle (mpctx=0x746050) at ../player/playloop.c:1020
#4  0x00000000004673b2 in mpctx_run_non_blocking (mpctx=mpctx@entry=0x746050, thread_fn=thread_fn@entry=0x462ec0 <open_demux_thread>, thread_arg=thread_arg@entry=0x7fffffffdf20) at ../player/misc.c:296
#5  0x0000000000464e96 in open_demux_async (stream=<optimized out>, mpctx=0x746050) at ../player/loadfile.c:889
#6  play_current_file (mpctx=0x746050) at ../player/loadfile.c:1004
#7  mp_play_files (mpctx=mpctx@entry=0x746050) at ../player/loadfile.c:1320
#8  0x000000000046685c in mpv_main (argc=<optimized out>, argv=<optimized out>) at ../player/main.c:542
#9  0x00007ffff2ebcec5 in __libc_start_main () from /lib/x86_64-linux-gnu/libc.so.6
#10 0x000000000040edbe in _start ()
(gdb) q
A debugging session is active.

    Inferior 1 [process 11580] will be killed.

Quit anyway? (y or n) y
kaefert@ultrablech ~ $
ghost commented 9 years ago

So this deadlocks because ffms is initialized inside of libavformat initialization, and the libavformat lock is not recursive.

qyot27 commented 9 years ago

So this deadlocks because ffms is initialized inside of libavformat initialization, and the libavformat lock is not recursive.

Which only causes this to manifest when FFMS2 is linked against a shared FFmpeg. As I mentioned in the bug report over on AvxSynth's side, I was able to build from HEAD of all the projects involved in this issue just about a week ago and never saw this problem arise (on an Athlon64 running Ubuntu 14.10 64-bit). But the difference is, I always build FFmpeg as static. Even a shared system FFmpeg would probably work fine if the copy FFMS2 was linked against was static, since there is no recursion in that case.

Not that that makes the situation any better.

avxsynth-testing commented 9 years ago

I'm pretty sure compatibility with FFmpeg avisynth.c demuxer is broken, not specifically for AvxSynth. Any case where mpv and ffms2 share a ffmpeg library will be broken because mpv is registering an avformat mutex which is recursively acquired first by avisynth.c and then by ffms2. The correct solution is to coordinate initialization of ffmpeg (avformat_network_init) between the libraries or to not register avformat mutexes to begin with.

EDIT: Perhaps the change to how avisynth.c loads the DLL in http://git.videolan.org/?p=ffmpeg.git;a=commit;h=f2ad1495f23376ce61542967f4fc14205f284d40 addresses this issue by making the ffmpeg build linked to ffms2 load in a separate namespace. I don't know enough POSIX to say for sure.