Avoid overhead from dynamic calls

[lachlansneff]

The executor boxes the futures, which adds significant overhead during every .poll call. Try parameterizing the executor on the type of future you pass in, and then also running Rust nightly, which recently removed the use of thread-local variables internally async fn implementations.

[XiangpengHao]

Hi @lachlansneff, thanks for your suggestions!

The Box<dyn Future<Output = u64> + 'inner> does issue two allocations, but only on task creation (https://github.com/XiangpengHao/async_bench/blob/master/src/executor.rs#L22), which in our case only happens const times (i.e. 4 times)

I tried the nightly version, the performance improved around 30% in our case.

I'm also wondering if we can reuse futures to reduce runtime allocation to zero...

Please let me know if my understanding is incorrect, I'm new to rust :)

[lachlansneff]

Your assumptions here are a little bit wrong. `Box<dyn Future

• ‘inner>` allocates once. And even though the allocations only occur once per task, there’s still actually a lot of overhead from having to poll a dynamic future. Having the futures be a concrete type that can get inlined will reduce overhead a lot.

On Wed, Jul 15, 2020 at 7:34 PM Xiangpeng Hao notifications@github.com wrote:

Hi @lachlansneff https://github.com/lachlansneff, thanks for your suggestions!

The Box<dyn Future

+ 'inner> does issue two allocations, but only on task creation ( https://github.com/XiangpengHao/async_bench/blob/master/src/executor.rs#L22), which in our case only happens const times (i.e. 4 times)

I tried the nightly version, the performance improved around 30% in our case.

I'm also wondering if we can reuse futures to reduce runtime allocation to zero...

Please let me know if my understanding is incorrect, I'm new to rust :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/XiangpengHao/async_bench/issues/1#issuecomment-659068483, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPIOK7F5UK3HAUXELCZL33R3Y4GTANCNFSM4OWZWTMA .

[XiangpengHao]

Oh yes, the poll will need to dereference the pointer, which can be costly. Thanks! I'm trying the async-task crate to see if I can do this easily.

[lachlansneff]

It’s actually not that the dereference is expensive, it’s that calling a function on a dynamic trait requires an actual call, which is expensive (in comparison to what we’re doing here). So you want to give the compiler enough information to inline poll.

I wrote an example on the rust playground a week or so ago to act as an example: https://play.rust-lang.org/?version=nightly&mode=debug&edition=2018&gist=3ddca1f690fd61d069e14d1d5f5d9fce

On Thu, Jul 16, 2020 at 12:49 PM Xiangpeng Hao notifications@github.com wrote:

Oh yes, the poll will need to dereference the pointer, which can be costly. Thanks! I'm trying the async-task crate to see if I can do this easily.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/XiangpengHao/async_bench/issues/1#issuecomment-659536903, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHPIOK4WUEPHRU7KWUBFVCTR34VSRANCNFSM4OWZWTMA .

[XiangpengHao]

Oh cool, I see. I didn't know we can write code like that :-) Thanks for letting know!