search

Jaded-Encoding-Thaumaturgy / vs-deband

Various debanding tools for VapourSynth
MIT License
15 stars 7 forks source link

Grainer.grain: Segmentation fault on temporal_average + size #99

Closed LightArrowsEXE closed 1 month ago

LightArrowsEXE commented 1 month ago

After complaining about my scripts randomly dying for a while now, I've traced it to this line of code.

https://github.com/Jaded-Encoding-Thaumaturgy/vs-deband/blob/f947f902daeed68da9cb2dd513b45268f0574afe/vsdeband/noise.py#L264

The grained VideoNode before that:

pre_avg_frames: VideoNode
        Format: YUV420P16
        Width: 1920
        Height: 1080
        Num Frames: 34068
        FPS: 24000/1001

The matrix:

[0.040986799232988945, 11.36450345862188, 332.11947410759177, 1023, 332.11947410759177, 11.36450345862188, 0.040986799232988945]

The annoying part is that this works on my local install, but not on another system, and I'm not sure how to deeper diagnose what the issue could be.

Same Vapoursynth version on both, so probably not related to std.AverageFrames?

$ python
Python 3.12.3 (tags/v3.12.3:f6650f9, Apr  9 2024, 14:05:25) [MSC v.1938 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from vstools import core
>>> core.version
<bound method Core.version of <vapoursynth.Core object at 0x000001C4F0A8FA10 core_version=R70, api_version=R4.1, num_threads=32, max_cache_size=4096, used_cache_size=0>>
$ python
Python 3.12.6 (tags/v3.12.6:a4a2d2b, Sep  6 2024, 20:11:23) [MSC v.1940 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from vstools import core
>>> core.version
<bound method Core.version of <vapoursynth.Core object at 0x000001F72AE8F880 core_version=R70, api_version=R4.1, num_threads=32, max_cache_size=4096, used_cache_size=0>>

Advice on how to better diagnose the exact issue would be great.

jsaowji commented 1 month ago

This is the backtrace from your sample from discord.

#0  std::_Hashtable<unsigned long long, std::pair<unsigned long long const, double>, std::allocator<std::pair<unsigned long long const, double> >, std::__detail::_Select1st, std::equal_to<unsigned long long>, std::hash<unsigned long long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_find_before_node (this=<optimized out>, __bkt=2128, __k=<optimized out>, __code=<optimized out>) at /usr/include/c++/14.2.1/bits/hashtable.h:2074
#1  std::_Hashtable<unsigned long long, std::pair<unsigned long long const, double>, std::allocator<std::pair<unsigned long long const, double> >, std::__detail::_Select1st, std::equal_to<unsigned long long>, std::hash<unsigned long long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::_M_find_node (this=<optimized out>, __key=<optimized out>, __c=13834490414079131136, __bkt=2128) at /usr/include/c++/14.2.1/bits/hashtable.h:822
#2  std::_Hashtable<unsigned long long, std::pair<unsigned long long const, double>, std::allocator<std::pair<unsigned long long const, double> >, std::__detail::_Select1st, std::equal_to<unsigned long long>, std::hash<unsigned long long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::find (__k=<synthetic pointer>: 13834490414079131136, this=<optimized out>) at /usr/include/c++/14.2.1/bits/hashtable.h:1738
#3  std::unordered_map<unsigned long long, double, std::hash<unsigned long long>, std::equal_to<unsigned long long>, std::allocator<std::pair<unsigned long long const, double> > >::find (this=<optimized out>, __x=<optimized out>) at /usr/include/c++/14.2.1/bits/unordered_map.h:877
#4  (anonymous namespace)::CustomZimgFilter::operator() (this=0x555556874280, x=-1.8739583333334622) at ../resize.cpp:798
#5  0x00007fffd673a23f in zimg::resize::compute_filter (f=..., src_dim=1828, dst_dim=1920, shift=0, width=<optimized out>) at ../zimg/src/zimg/resize/filter.cpp:289
#6  0x00007fffd673c454 in zimg::resize::ResizeImplBuilder::create (this=this@entry=0x7fff9f3fe660) at ../zimg/src/zimg/resize/resize_impl.cpp:250
#7  0x00007fffd673b0db in zimg::resize::ResizeConversion::create (this=0x7fff9f3fe780) at ../zimg/src/zimg/resize/resize.cpp:77
#8  0x00007fffd6734e2c in zimg::graph::GraphBuilder::impl::resize_plane (this=this@entry=0x7fff90075eb0, target=..., params=..., observer=..., mask=mask@entry=..., p=p@entry=0) at ../zimg/src/zimg/graph/graphbuilder.cpp:819
#9  0x00007fffd6735adc in zimg::graph::GraphBuilder::impl::connect_plane (this=this@entry=0x7fff90075eb0, target=..., params=..., observer=..., mode=mode@entry=zimg::graph::GraphBuilder::impl::ConnectMode::LUMA, reinterpret_range=reinterpret_range@entry=true) at ../zimg/src/zimg/graph/graphbuilder.cpp:957
#10 0x00007fffd6736eba in zimg::graph::GraphBuilder::impl::connect_color_channels_planar (this=0x7fff90075eb0, target=..., params=..., observer=..., reinterpret_range=true) at ../zimg/src/zimg/graph/graphbuilder.cpp:972
#11 zimg::graph::GraphBuilder::impl::connect_color_channels (this=0x7fff90075eb0, target=..., params=..., observer=...) at ../zimg/src/zimg/graph/graphbuilder.cpp:1018
#12 0x00007fffd6737f91 in zimg::graph::GraphBuilder::impl::connect_internal (this=this@entry=0x7fff90075eb0, target=..., params=..., observer=...) at ../zimg/src/zimg/graph/graphbuilder.cpp:1072
#13 0x00007fffd6733473 in zimg::graph::GraphBuilder::impl::connect (this=0x7fff90075eb0, target=..., params=..., observer=...) at ../zimg/src/zimg/graph/graphbuilder.cpp:1118
#14 zimg::graph::GraphBuilder::connect (this=0x7fff9f3ff380, target=..., params=params@entry=0x7fff9f3ff3f0, observer=0x7fff9f3fef28, observer@entry=0x0) at ../zimg/src/zimg/graph/graphbuilder.cpp:1245
#15 0x00007fffd670d52b in (anonymous namespace)::vszimg::graph_data::graph_data (this=0x7fff90075dd0, src_format=..., dst_format=..., params=..., filters=0x5555555d0808) at ../resize.cpp:883
#16 std::_Construct<(anonymous namespace)::vszimg::graph_data, (anonymous namespace)::vsrz_image_format const&, (anonymous namespace)::vsrz_image_format const&, (anonymous namespace)::filter_graph_builder_params&, std::unique_ptr<zimg::resize::Filter, std::default_delete<zimg::resize::Filter> > (&) [2]> (__p=0x7fff90075dd0) at /usr/include/c++/14.2.1/bits/stl_construct.h:119
#17 std::allocator_traits<std::allocator<void> >::construct<(anonymous namespace)::vszimg::graph_data, (anonymous namespace)::vsrz_image_format const&, (anonymous namespace)::vsrz_image_format const&, (anonymous namespace)::filter_graph_builder_params&, std::unique_ptr<zimg::resize::Filter, std::default_delete<zimg::resize::Filter> > (&) [2]> (__p=0x7fff90075dd0) at /usr/include/c++/14.2.1/bits/alloc_traits.h:657
#18 std::_Sp_counted_ptr_inplace<(anonymous namespace)::vszimg::graph_data, std::allocator<void>, (__gnu_cxx::_Lock_policy)2>::_Sp_counted_ptr_inplace<(anonymous namespace)::vsrz_image_format const&, (anonymous namespace)::vsrz_image_format const&, (anonymous namespace)::filter_graph_builder_params&, std::unique_ptr<zimg::resize::Filter, std::default_delete<zimg::resize::Filter> > (&) [2]> (this=0x7fff90075dc0, __a=...) at /usr/include/c++/14.2.1/bits/shared_ptr_base.h:607
#19 std::__shared_count<(__gnu_cxx::_Lock_policy)2>::__shared_count<(anonymous namespace)::vszimg::graph_data, std::allocator<void>, (anonymous namespace)::vsrz_image_format const&, (anonymous namespace)::vsrz_image_format const&, (anonymous namespace)::filter_graph_builder_params&, std::unique_ptr<zimg::resize::Filter, std::default_delete<zimg::resize::Filter> > (&) [2]> (__a=..., this=<optimized out>, __p=<optimized out>) at /usr/include/c++/14.2.1/bits/shared_ptr_base.h:969
#20 std::__shared_ptr<(anonymous namespace)::vszimg::graph_data, (__gnu_cxx::_Lock_policy)2>::__shared_ptr<std::allocator<void>, (anonymous namespace)::vsrz_image_format const&, (anonymous namespace)::vsrz_image_format const&, (anonymous namespace)::filter_graph_builder_params&, std::unique_ptr<zimg::resize::Filter, std::default_delete<zimg::resize::Filter> > (&) [2]> (this=<optimized out>, __tag=...) at /usr/include/c++/14.2.1/bits/shared_ptr_base.h:1713
#21 std::shared_ptr<(anonymous namespace)::vszimg::graph_data>::shared_ptr<std::allocator<void>, (anonymous namespace)::vsrz_image_format const&, (anonymous namespace)::vsrz_image_format const&, (anonymous namespace)::filter_graph_builder_params&, std::unique_ptr<zimg::resize::Filter, std::default_delete<zimg::resize::Filter> > (&) [2]> (this=<optimized out>, __tag=...) at /usr/include/c++/14.2.1/bits/shared_ptr.h:463
#22 std::make_shared<(anonymous namespace)::vszimg::graph_data, (anonymous namespace)::vsrz_image_format const&, (anonymous namespace)::vsrz_image_format const&, (anonymous namespace)::filter_graph_builder_params&, std::unique_ptr<zimg::resize::Filter, std::default_delete<zimg::resize::Filter> > (&) [2]> () at /usr/include/c++/14.2.1/bits/shared_ptr.h:1008
#23 (anonymous namespace)::vszimg::get_graph_data (this=0x5555555d0770, src_format=..., dst_format=...) at ../resize.cpp:1098
#24 (anonymous namespace)::vszimg::real_get_frame (this=this@entry=0x5555555d0770, src_frame=src_frame@entry=0x7fff900ea690, core=core@entry=0x555555e3c3c0, vsapi=vsapi@entry=0x7ffff6b671c0) at ../resize.cpp:1222
#25 0x00007fffd670f0cf in (anonymous namespace)::vszimg::get_frame (frameData=<optimized out>, this=0x5555555d0770, n=<optimized out>, activationReason=<optimized out>, frameCtx=0x7fffa00ff710, core=0x555555e3c3c0, vsapi=0x7ffff6b671c0) at ../resize.cpp:1285
#26 (anonymous namespace)::vszimg::static_get_frame (n=<optimized out>, activationReason=<optimized out>, instanceData=0x5555555d0770, frameData=<optimized out>, frameCtx=0x7fffa00ff710, core=0x555555e3c3c0, vsapi=0x7ffff6b671c0) at ../resize.cpp:1337
#27 0x00007ffff6a58aa3 in ?? () from /usr/lib/libvapoursynth.so
#28 0x00007ffff66e1cf4 in std::execute_native_thread_routine (__p=0x7fffa00fe930) at /usr/src/debug/gcc/gcc/libstdc++-v3/src/c++11/thread.cc:104
#29 0x00007ffff76a339d in ?? () from /usr/lib/libc.so.6
#30 0x00007ffff772849c in ?? () from /usr/lib/libc.so.6

I think its resize2 related. Smaller repro but still crahes same way for me. Just much less around ~1x in 15 runs or so.

from vskernels import Lanczos
from vstools import core,vs
from vstools import clip_async_render

src = core.std.BlankClip(format=vs.YUV420P8,width=1920*2,height=1080*2,length=200)
grain = Lanczos().scale(src,width=100,height=100)
clip_async_render(grain)

It may be threading releated since putting a lock in CustomZimgFilter::operator() seems to solve it, but i dont know zimg or resize2 so im not going to speculate.