Sporatic freezes due to threading

[vi]

Sometimes it just locks up and fails to progress, keeping CPU (one thread) busy.

Just trying again (another seed?) seems to be resolving the issue.

$ texture-synthesis  --out=z.png --k-neighs 50 --m-rand 50   --tiling generate q.png    && tile2x2 z.png zz.png
[00:00:27] ###########################------------- 66%
 stage   6 ######################################-- 95%

^C
$ texture-synthesis  --out=z.png --k-neighs 50 --m-rand 50   --tiling generate q.png    && tile2x2 z.png zz.png
[00:00:01] ##-------------------------------------- 4%
[00:00:25] ######################################## 100%
 stage   6 ######################################## 100%
$

Using low --k-neighs increases probability of fail. Explicitly specifying --seed seems to have no effect:

$ texture-synthesis  --out=z.png --k-neighs 10 --m-rand 50   --tiling --seed 2 generate q.png 
[00:00:07] ###########################------------- 66%
 stage   6 ######################################-- 95%
^C
$ texture-synthesis  --out=z.png --k-neighs 10 --m-rand 50   --tiling --seed 2 generate q.png 
[00:00:07] ######################################## 100%
 stage   6 ######################################## 100%
$ texture-synthesis  --out=z.png --k-neighs 10 --m-rand 50   --tiling --seed 2 generate q.png 
[00:00:07] ######################################## 100%
 stage   6 ######################################## 100%
$ texture-synthesis  --out=z.png --k-neighs 10 --m-rand 50   --tiling --seed 2 generate q.png 
[00:00:07] ###########################------------- 66%
 stage   6 ######################################-- 95%
^C
$

Workaround: -t 1 to turn off threading. No repro even with --k-neighs 1.

[h3r2tic]

Thank you for reporting this! Knowing that a low value of k-neighs increases its likelihood is very useful. I've had it happen a few times before, but it seemed extremely rare. I'll try it on a machine with a ton of cores with a low k-neighs, and see if that gets me anywhere.

[vi]

Backtraces when it is stuck:

(gdb) thread apply all bt

Thread 2 (Thread 0x7f6611bad700 (LWP 17916)):
#0  futex_wait_cancelable (private=0, expected=0, futex_word=0x55bff8d6a578) at ../sysdeps/unix/sysv/linux/futex-internal.h:88
#1  __pthread_cond_wait_common (abstime=0x0, mutex=0x55bff8d6b900, cond=0x55bff8d6a550) at pthread_cond_wait.c:502
#2  __pthread_cond_wait (cond=0x55bff8d6a550, mutex=0x55bff8d6b900) at pthread_cond_wait.c:655
#3  0x000055bff6e98c33 in std::sys::unix::condvar::Condvar::wait () at src/libstd/sys/unix/condvar.rs:71
#4  std::sys_common::condvar::Condvar::wait () at src/libstd/sys_common/condvar.rs:41
#5  std::sync::condvar::Condvar::wait () at src/libstd/sync/condvar.rs:204
#6  std::thread::park () at src/libstd/thread/mod.rs:911
#7  0x000055bff6e9f3f2 in std::sync::mpsc::blocking::WaitToken::wait () at src/libstd/sync/mpsc/blocking.rs:71
#8  0x000055bff6d62efd in std::sync::mpsc::shared::Packet<T>::recv ()
#9  0x000055bff6d57c94 in std::sync::mpsc::Receiver<T>::recv ()
#10 0x000055bff6d52e7c in indicatif::progress::MultiProgress::join_impl ()
#11 0x000055bff6d52d3b in indicatif::progress::MultiProgress::join ()
#12 0x000055bff6d4766c in std::sys_common::backtrace::__rust_begin_short_backtrace ()
#13 0x000055bff6d39226 in std::panicking::try::do_call ()
#14 0x000055bff6ea5f9a in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#15 0x000055bff6d4c8a6 in core::ops::function::FnOnce::call_once{{vtable-shim}} ()
#16 0x000055bff6e9828f in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/521d78407471cb78e9bbf47160f6aa23047ac499/src/liballoc/boxed.rs:922
#17 0x000055bff6ea5360 in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once () at /rustc/521d78407471cb78e9bbf47160f6aa23047ac499/src/liballoc/boxed.rs:922
#18 std::sys_common::thread::start_thread () at src/libstd/sys_common/thread.rs:13
#19 std::sys::unix::thread::Thread::new::thread_start () at src/libstd/sys/unix/thread.rs:79
#20 0x00007f6612465fa3 in start_thread (arg=<optimized out>) at pthread_create.c:486
#21 0x00007f661257d4cf in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95

Thread 1 (Thread 0x7f66122c0240 (LWP 17914)):
#0  0x000055bff6dc2cec in std::panicking::try::do_call ()
#1  0x000055bff6ea5f9a in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#2  0x000055bff6dc4648 in crossbeam_utils::thread::scope ()
#3  0x000055bff6db5946 in texture_synthesis::multires_stochastic_texture_synthesis::Generator::main_resolve_loop ()
#4  0x000055bff6dbd5f3 in texture_synthesis::Session::run ()
#5  0x000055bff6d45886 in texture_synthesis::main ()
#6  0x000055bff6d4c323 in std::rt::lang_start::{{closure}} ()
#7  0x000055bff6ea2c23 in std::rt::lang_start_internal::{{closure}} () at src/libstd/rt.rs:49
#8  std::panicking::try::do_call () at src/libstd/panicking.rs:296
#9  0x000055bff6ea5f9a in __rust_maybe_catch_panic () at src/libpanic_unwind/lib.rs:80
#10 0x000055bff6ea37ad in std::panicking::try () at src/libstd/panicking.rs:275
#11 std::panic::catch_unwind () at src/libstd/panic.rs:394
#12 std::rt::lang_start_internal () at src/libstd/rt.rs:48
#13 0x000055bff6d46612 in main ()
(gdb)

[h3r2tic]

I think I have a fix for both this and #24. If you have a somewhat reliable repro case, please give it a spin (see the pull request above).

[JCBrouwer]

I've still been getting sporadic freezing with versions since #26. Both on the 0.5.0 release and built from the most recent master (202fde1099c712ced6d9a0854df7d45cbb52f1b8) with Rust nightly version 1.37.0 (eae3437df 2019-08-13) on Ubuntu 18.04 (built with cargo build --release and copied to /usr/bin).

I'm not sure if it's threading related and don't have specific reproduction cases, but it's been happening with many different images when generating with guides. An example of the commands I've been running and encountered a freeze with:

texture-synthesis -o output.png --seed 123 --rand-init 321 --alpha 0.3 --in-size 1000 --out-size 1500x2000 generate --target-guide target.jpg --guides guide1.jpg guide2.jpg guide3.jpg guide4.jpg -- tex1.jpg tex2.jpg tex3.jpg tex4.jpg

A nice workaround for long-running bash scripts that were hanging (the above command was taking about 4.5 min on average):

until timeout 300 texture-synthesis [rest of command args here]; do sleep 301; done