dwango / fibers-rs

A Rust library to execute a number of lightweight asynchronous tasks (a.k.a, fibers) based on futures and mio.
MIT License
182 stars 11 forks source link

Add new tasks after the creation of an executor #13

Closed astonbitecode closed 5 years ago

astonbitecode commented 5 years ago

Hi,

Is it somehow possible for an Executor to be shared between threads and also be used for adding new tasks during runtime?

I am thinking of something like:

// Create a shared executor
let mut executor_mutex = Arc::new(Mutex::new(ThreadPoolExecutor::new().unwrap()));

// Clone it
let ce = executor_mutex.clone();

// A different thread, at some poipnt spawns new tasks on the executor
thread::spawn(move || {
    let mut executor = ce.lock().unwrap();
    executor.spawn(futures::lazy(|| { println!("Hello"); Ok(())} ));
});

let executor = executor_mutex.lock().unwrap();

// Run the executor
// Not possible, since the run method consumes the executor
executor.run();

I see that there is the run_once, but I suppose that this blocks until all the units that it includes finish. This would make the executor practically unavailable until then I guess.

On the other hand, I could just create one new executor whenever I wanted to execute some tasks, but then the occupied threads on the system would increase drastically if there are a lot of tasks to be executed and these tasks are long-lived...

sile commented 5 years ago

Thank you for your question. There are two ways for handling such situation.

(1) Send handles instead of executors

Executor trait provides handle method. By calling this method, you can create any number of handles that can be sent to separate threads. Since handles implement Spawn trait, you can spawn futures via a handle without executors. Running examples that use handles are found in fibers-rs/examples/ directory.

(2) Use fibers_global crate

This crate manages the global ThreadPoolExecutor instead of you, and provides fibers_global::spawn method that can be called anywhere within your code. fibers_global is convenient if you don't need advanced features such as managing multiple executors in a single program.

astonbitecode commented 5 years ago

Thank you for the suggestions. I didn't know about the fibers_global crate.

I will give it a try, I don't have more complex cases for now.