Closed slawlor closed 11 months ago
Attention: 4 lines
in your changes are missing coverage. Please review.
Comparison is base (
1ae8d86
) 79.20% compared to head (20aae31
) 79.23%.
:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.
Running our benchmark suite we can measure somewhat the performance delta between tokio
and async-std
, however these only measure very basic operations and not all functionality in the crate.
Starting with tokio
, we have
$ cargo bench -p ractor
...
Creation of 100 actors time: [402.38 µs 414.38 µs 431.41 µs]
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) low mild
3 (3.00%) high mild
3 (3.00%) high severe
Waiting on 100 actors to process first message
time: [953.14 µs 1.0024 ms 1.0515 ms]
Found 12 outliers among 100 measurements (12.00%)
8 (8.00%) high mild
4 (4.00%) high severe
Waiting on 1000 actors to process first message
time: [9.8127 ms 9.9778 ms 10.148 ms]
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild
Waiting on 100000 messages to be processed
time: [23.755 ms 23.875 ms 24.001 ms]
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Then running again with the async-std
runtime
$ cargo bench -p ractor --features async-std
Creation of 100 actors time: [470.98 µs 482.73 µs 500.55 µs]
change: [+15.865% +18.363% +21.029%] (p = 0.00 < 0.05)
Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) low severe
1 (1.00%) low mild
5 (5.00%) high mild
3 (3.00%) high severe
Benchmarking Creation of 10000 actors: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 5.1s, or reduce sample count to 90.
Creation of 10000 actors
time: [54.189 ms 55.454 ms 56.750 ms]
change: [+25.379% +29.767% +34.366%] (p = 0.00 < 0.05)
Performance has regressed.
Benchmarking Waiting on 100 actors to process first message: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 2892.0s, or reduce sample count to 10.
Waiting on 100 actors to process first message
time: [700.84 µs 750.61 µs 799.74 µs]
change: [-29.340% -21.688% -14.527%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
5 (5.00%) low mild
3 (3.00%) high mild
Waiting on 1000 actors to process first message
time: [8.9662 ms 9.1759 ms 9.4057 ms]
change: [-10.577% -8.0368% -5.1212%] (p = 0.00 < 0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
3 (3.00%) high mild
3 (3.00%) high severe
This shows that actor creation is more expensive in async-std
but trivial message processing schedules faster. We likely need more complex performance tests to compare the runtime functionalities better, but this is a start showing reasonable performance in either runtime.
This PR adds support for the
async-std
runtime as an alternative to thetokio
runtime. It can be enabled with theasync-std
feature.Note:
ractor
uses some core primitives or functionality of structs fromtokio
which required some wrapping to support inasync-std
. Specifically:tokio
'sJoinHandle
has functionality to determine if aJoinHandle
is finished as well as to abort it without consuming the handle. These required some wrapping in ourasync_std_primatives
module to support similar functionality without great rewrites ofractor
. Basic functionality test coverage is added for our customJoinHandle
implementation.tokio
has aJoinSet
which has some minor syntactical changes from the standardFuturesUnordered
from the futures crate which we can leverage in its stead. Again a small wrapper was added.tokio
has anInterval
which is used to fix #57, this doesn't exist inasync-std
aside from a basicsleep
implementation, so we had to add a custom version of it.