search

tikv / yatp

Yet another thread pool in rust for both callbacks or futures.
Apache License 2.0
135 stars 32 forks source link

Update tokio to 0.3 #48

Closed liufuyang closed 3 years ago

liufuyang commented 3 years ago

Tokio 0.3 is out for some days, might be interesting to see the benchmark with it.

But my change here seems still have some issues such for that ping_pong bench:

Benchmarking ping_pong/tokio/512: Collecting 100 samples in estimated 6.5420 s (20k iterations)thread 'tokio-runtime-worker' panicked at 'Cannot drop a runtime in a context where blocking is not allowed. This happens when a runtime is dropped from within an asynchronous context.', /Users/fuyangl/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-0.3.3/src/runtime/blocking/shutdown.rs:51:21
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

thread 'tokio-runtime-worker' has overflowed its stack
fatal runtime error: stack overflow
error: process didn't exit successfully: `/Users/fuyangl/Workspace/yatp/target/release/deps/ping_pong-4c2752ff3e0de76a ping_pong --bench` (signal: 6, SIGABRT: process abort signal)

Need to solve this first I suppose.

liufuyang commented 3 years ago

Looks like they just added the handle back in 0.3.4, so switch onto using handle solves the above issue.

BusyJay commented 3 years ago

What does the benchmark results look like?

liufuyang commented 3 years ago

I would say not much changed? But I can try double check again. I have run it a few times before but due to the computer is in different load situation so the results seems not that stable.

Master branch:

fuyangl@Fuyangs-MacBook-Pro:~/Workspace/yatp
(master) 🦀 👉  cargo update
    Updating crates.io index
      Adding bytes v0.5.6
    Updating cc v1.0.63 -> v1.0.65
    Updating const-random v0.1.11 -> v0.1.12
    Updating const-random-macro v0.1.11 -> v0.1.12
      Adding crunchy v0.2.2
    Removing pin-project-lite v0.2.0
      Adding tiny-keccak v2.0.2
    Updating tokio v0.3.4 -> v0.2.23
fuyangl@Fuyangs-MacBook-Pro:~/Workspace/yatp
(master) 🦀 👉  cargo bench chained_spawn
  Downloaded crunchy v0.2.2
  Downloaded const-random-macro v0.1.12
  Downloaded tiny-keccak v2.0.2
  Downloaded const-random v0.1.12
  Downloaded 4 crates (36.2 KB) in 0.70s
   Compiling crunchy v0.2.2
   Compiling tiny-keccak v2.0.2
   Compiling lazy_static v1.4.0
   Compiling const-random-macro v0.1.12
   Compiling const-random v0.1.12
   Compiling ahash v0.3.8
   Compiling dashmap v3.11.10
   Compiling yatp v0.0.1 (/Users/fuyangl/Workspace/yatp)
    Finished bench [optimized + debuginfo] target(s) in 28.79s
     Running target/release/deps/yatp-2fd1b90802b4b9dc

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 30 filtered out

     Running target/release/deps/chained_spawn-17115ab4b033a375
chained_spawn/yatp::future/256
                        time:   [283.13 us 283.82 us 284.44 us]
                        change: [-1.3445% +0.4349% +1.7849%] (p = 0.64 > 0.05)
                        No change in performance detected.
chained_spawn/yatp::callback/256
                        time:   [49.128 us 49.565 us 50.024 us]
                        change: [-2.5441% -1.0125% +0.5612%] (p = 0.20 > 0.05)
                        No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild
chained_spawn/yatp::future::multilevel/256
                        time:   [373.23 us 376.23 us 379.06 us]
                        change: [+1.1765% +2.6997% +4.2310%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 13 outliers among 100 measurements (13.00%)
  3 (3.00%) low severe
  1 (1.00%) low mild
  5 (5.00%) high mild
  4 (4.00%) high severe
chained_spawn/tokio/256 time:   [51.535 us 51.968 us 52.442 us]
                        change: [-3.6036% -1.3948% +1.1505%] (p = 0.25 > 0.05)
                        No change in performance detected.
Found 9 outliers among 100 measurements (9.00%)
  4 (4.00%) high mild
  5 (5.00%) high severe
chained_spawn/async-std/256
                        time:   [273.29 us 274.16 us 275.12 us]
                        change: [+4.0052% +4.8032% +5.6592%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  1 (1.00%) low mild
  9 (9.00%) high mild
chained_spawn/yatp::future/512
                        time:   [621.53 us 622.78 us 624.07 us]
                        change: [+1.0426% +1.6997% +2.3829%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
  1 (1.00%) low severe
  4 (4.00%) high mild
  4 (4.00%) high severe
chained_spawn/yatp::callback/512
                        time:   [101.87 us 104.41 us 107.13 us]
                        change: [+6.6833% +9.0876% +11.466%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild
chained_spawn/yatp::future::multilevel/512
                        time:   [764.04 us 767.24 us 770.72 us]
                        change: [+1.2667% +2.3499% +3.4305%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe
chained_spawn/tokio/512 time:   [96.478 us 98.399 us 100.52 us]
                        change: [-0.4080% +1.7201% +3.8446%] (p = 0.11 > 0.05)
                        No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe
chained_spawn/async-std/512
                        time:   [521.66 us 528.25 us 534.80 us]
                        change: [-0.8178% +0.5473% +1.9176%] (p = 0.44 > 0.05)
                        No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
  6 (6.00%) low mild
  2 (2.00%) high mild
Benchmarking chained_spawn/yatp::future/1024: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.1s, enable flat sampling, or reduce sample count to 60.
chained_spawn/yatp::future/1024
                        time:   [1.1856 ms 1.1897 ms 1.1940 ms]
                        change: [+2.9975% +3.5482% +4.1501%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
chained_spawn/yatp::callback/1024
                        time:   [172.81 us 174.38 us 176.10 us]
                        change: [-3.6202% -0.6024% +2.2172%] (p = 0.70 > 0.05)
                        No change in performance detected.
Found 10 outliers among 100 measurements (10.00%)
  7 (7.00%) high mild
  3 (3.00%) high severe
Benchmarking chained_spawn/yatp::future::multilevel/1024: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.6s, enable flat sampling, or reduce sample count to 50.
chained_spawn/yatp::future::multilevel/1024
                        time:   [1.4619 ms 1.4691 ms 1.4757 ms]
                        change: [-0.7171% +1.1749% +3.0777%] (p = 0.24 > 0.05)
                        No change in performance detected.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild
chained_spawn/tokio/1024
                        time:   [184.96 us 187.50 us 190.21 us]
                        change: [+1.5647% +4.3047% +7.0295%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking chained_spawn/async-std/1024: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.2s, enable flat sampling, or reduce sample count to 60.
chained_spawn/async-std/1024
                        time:   [1.0241 ms 1.0291 ms 1.0345 ms]
                        change: [+4.0108% +4.9963% +6.0549%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild

     Running target/release/deps/ping_pong-7f3958d9f586588d
     Running target/release/deps/spawn_many-fddf635e892e168f
     Running target/release/deps/yield_many-c19f9c0311de07ef

Tokio 0.3 branch:

fuyangl@Fuyangs-MacBook-Pro:~/Workspace/yatp
(master) 🦀 👉  git checkout update-tokio-0.3
Switched to branch 'update-tokio-0.3'
fuyangl@Fuyangs-MacBook-Pro:~/Workspace/yatp
(update-tokio-0.3) 🦀 👉  cargo update
    Updating crates.io index
    Removing bytes v0.5.6
      Adding pin-project-lite v0.2.0
    Updating tokio v0.2.23 -> v0.3.4
fuyangl@Fuyangs-MacBook-Pro:~/Workspace/yatp
(update-tokio-0.3) 🦀 👉  cargo bench chained_spawn
   Compiling yatp v0.0.1 (/Users/fuyangl/Workspace/yatp)
    Finished bench [optimized + debuginfo] target(s) in 25.25s
     Running target/release/deps/yatp-c0d427f8bb85acae

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 30 filtered out

     Running target/release/deps/chained_spawn-84485cf4d0e073c5
chained_spawn/yatp::future/256
                        time:   [272.96 us 275.95 us 280.74 us]
                        change: [-3.8547% -3.2043% -2.3134%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe
chained_spawn/yatp::callback/256
                        time:   [48.635 us 49.200 us 49.830 us]
                        change: [-1.4524% +0.0778% +1.8255%] (p = 0.93 > 0.05)
                        No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe
chained_spawn/yatp::future::multilevel/256
                        time:   [371.81 us 374.43 us 376.90 us]
                        change: [-1.3381% +0.1619% +1.6583%] (p = 0.84 > 0.05)
                        No change in performance detected.
Found 13 outliers among 100 measurements (13.00%)
  6 (6.00%) low mild
  3 (3.00%) high mild
  4 (4.00%) high severe
chained_spawn/tokio/256 time:   [52.065 us 52.824 us 53.724 us]
                        change: [+1.8639% +4.9626% +8.0511%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe
chained_spawn/async-std/256
                        time:   [264.74 us 266.79 us 269.56 us]
                        change: [-3.8414% -2.9914% -1.9757%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe
chained_spawn/yatp::future/512
                        time:   [622.20 us 630.22 us 639.62 us]
                        change: [-0.2774% +0.5047% +1.4423%] (p = 0.26 > 0.05)
                        No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) high mild
  3 (3.00%) high severe
chained_spawn/yatp::callback/512
                        time:   [92.351 us 95.535 us 99.473 us]
                        change: [-8.9101% -6.5690% -3.9338%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 14 outliers among 100 measurements (14.00%)
  10 (10.00%) high mild
  4 (4.00%) high severe
chained_spawn/yatp::future::multilevel/512
                        time:   [765.39 us 767.60 us 770.04 us]
                        change: [-0.9392% +0.0175% +0.9048%] (p = 0.97 > 0.05)
                        No change in performance detected.
Found 6 outliers among 100 measurements (6.00%)
  1 (1.00%) high mild
  5 (5.00%) high severe
chained_spawn/tokio/512 time:   [94.091 us 95.085 us 96.258 us]
                        change: [-4.1106% -2.4469% -0.7435%] (p = 0.01 < 0.05)
                        Change within noise threshold.
Found 10 outliers among 100 measurements (10.00%)
  9 (9.00%) high mild
  1 (1.00%) high severe
chained_spawn/async-std/512
                        time:   [521.82 us 524.82 us 528.03 us]
                        change: [+0.1000% +1.2630% +2.4837%] (p = 0.05 < 0.05)
                        Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe
Benchmarking chained_spawn/yatp::future/1024: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.9s, enable flat sampling, or reduce sample count to 60.
chained_spawn/yatp::future/1024
                        time:   [1.1769 ms 1.1810 ms 1.1851 ms]
                        change: [-1.4296% -0.8239% -0.1853%] (p = 0.01 < 0.05)
                        Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe
chained_spawn/yatp::callback/1024
                        time:   [174.37 us 176.21 us 178.37 us]
                        change: [+0.5439% +2.7060% +4.7737%] (p = 0.01 < 0.05)
                        Change within noise threshold.
Benchmarking chained_spawn/yatp::future::multilevel/1024: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.5s, enable flat sampling, or reduce sample count to 50.
chained_spawn/yatp::future::multilevel/1024
                        time:   [1.4703 ms 1.4784 ms 1.4855 ms]
                        change: [-1.1455% +0.7567% +2.6462%] (p = 0.45 > 0.05)
                        No change in performance detected.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
chained_spawn/tokio/1024
                        time:   [178.82 us 180.26 us 181.88 us]
                        change: [-6.6710% -4.0585% -0.7489%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
  6 (6.00%) high mild
  2 (2.00%) high severe
Benchmarking chained_spawn/async-std/1024: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.1s, enable flat sampling, or reduce sample count to 70.
chained_spawn/async-std/1024
                        time:   [1.0112 ms 1.0174 ms 1.0239 ms]
                        change: [-3.0356% -1.9371% -0.8479%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
  4 (4.00%) high mild

     Running target/release/deps/ping_pong-f6061d60608986ff
     Running target/release/deps/spawn_many-fa6079e7fdadcca8
     Running target/release/deps/yield_many-48bfc25db77c55d4
liufuyang commented 3 years ago

@BusyJay Another thing I encountered is that, while I saw tokio have some default stack_size for the task is 2Mb https://docs.rs/tokio/0.2.1/tokio/runtime/struct.Builder.html#method.thread_stack_size, so I thought I should try to set the stack size as well to see what happens.

yatp::Builder::new("chained_spawn").stack_size(2048).build_callback_pool();

Question one is, from the doc of stack_size I cannot tell the unit of the value. Perhaps 2048 is too large or I get this while running as the code above:

Benchmarking chained_spawn/yatp::callback/256: Warming up for 3.0000 s
thread 'chained_spawn-0' has overflowed its stack
fatal runtime error: stack overflow

thread 'chained_spawn-1' has overflowed its stack
fatal runtime error: stack overflow

thread 'chained_spawn-3' has overflowed its stack
fatal runtime error: stack overflow

thread 'chained_spawn-5' has overflowed its stack
fatal runtime error: stack overflow
error: process didn't exit successfully: `/Users/fuyangl/Workspace/yatp/target/release/deps/chained_spawn-84485cf4d0e073c5 chained_spawn --bench` (signal: 6, SIGABRT: process abort signal)

Then I tried to change from 2048 to 2 but still getting the same error. So I am not sure whether that is a bug or? 🤔

BusyJay commented 3 years ago

It's the same as https://doc.rust-lang.org/std/thread/struct.Builder.html#method.stack_size, the unit is byte.

liufuyang commented 3 years ago

I see. Then why using it triggering those stack overflow errors? That seems not a sound behavior or? I tried to set .thread_stack_size(2048) on tokio and there is no problem on that.

liufuyang commented 3 years ago

Perhaps we can merge this and then I open an issue page about it?