Reboot leader service failed due to "Resource temporarily unavailable"

[luhuimao]

Repro steps:

1. leader service or validator service crashed , detail see in https://github.com/bitconch/bus/issues/124 .
2. reboot leader or validator service
3. hit the issue.

Call Stack: 1 thread 'main' panicked at 'called Result::unwrap() on an Err value: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }', src/libcore/result.rs:997:5 [2019-07-29T02:37:07.184736898Z INFO soros::replay_stage] @1564367827184 locktower duration: 0 len: 1 weights: [1008] [2019-07-29T02:37:07.184953044Z INFO soros::poh_recorder] reset poh from: 8eoBiorKPY7mqx76tDNYo4HhPqRyFurfM4iB2tQY28J3,520551 to: 8eoBiorKPY7mqx76tDNYo4HhPqRyFurfM4iB2tQY28J3,520551 [2019-07-29T02:37:07.184975092Z WARN soros::cluster_info] CmkGmp1J1jCv5YHKW7WDvyrwhNnkNQSwaDJaU2rQ1t61: LEADER_UPDATE TO CmkGmp1J1jCv5YHKW7WDvyrwhNnkNQSwaDJaU2rQ1t61 from CmkGmp1J1jCv5YHKW7WDvyrwhNnkNQSwaDJaU2rQ1t61 thread 'thread 'soros-jsonrpchttp.worker3' panicked at '' panicked at 'failed to spawn worker thread: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }failed to spawn worker thread: Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }', ', /root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.15/src/pool/mod.rs:/root/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-threadpool-0.1.15/src/pool/mod.rs389::38913: 13 stack backtrace: 0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39 1: std::sys_common::backtrace::_print at src/libstd/sys_common/backtrace.rs:70 2: std::panicking::default_hook::{{closure}} at src/libstd/sys_common/backtrace.rs:58 at src/libstd/panicking.rs:200 3: std::panicking::default_hook at src/libstd/panicking.rs:215 4: soros_metrics::metrics::set_panic_hook::{{closure}}::{{closure}} 5: std::panicking::rust_panic_with_hook at src/libstd/panicking.rs:482 6: std::panicking::continue_panic_fmt at src/libstd/panicking.rs:385 7: rust_begin_unwind at src/libstd/panicking.rs:312 8: core::panicking::panic_fmt at src/libcore/panicking.rs:85 9: core::result::unwrap_failed 10: soros::streamer::receiver 11: <core::iter::adapters::Map<I, F> as core::iter::traits::iterator::Iterator>::fold 12: <core::iter::adapters::chain::Chain<A, B> as core::iter::traits::iterator::Iterator>::fold 13: <alloc::vec::Vec as alloc::vec::SpecExtend<T, I>>::from_iter 14: soros::fetch_stage::FetchStage::new_with_sender 15: soros::tpu::Tpu::new 16: soros::fullnode::Fullnode::new 17: soros_fullnode::main 18: std::rt::lang_start::{{closure}}