Closed P403n1x87 closed 1 year ago
Supersedes #182.
Running Austin benchmarks with Python 3.10.12
Wall time [sampling interval: 1]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
====================================================================
Sample Rate Saturation Error Rate Sampling Speed
--------------------------------------------------------------------
3.2.0 21600 ± 600 1 ± 0 2.49e-05 ± 5e-07 74 ± 2
3.3.0 102000 ± 3000 1 ± 0 4e-06 ± 2e-06 15.4 ± 0.5
3.4.1 103000 ± 3000 1 ± 0 1.3e-05 ± 6e-06 15.3 ± 0.5
dev 105000 ± 2000 1 ± 0 1.4e-05 ± 7e-06 14.9 ± 0.3
====================================================================
Wall time [sampling interval: 10]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
========================================================================
Sample Rate Saturation Error Rate Sampling Speed
------------------------------------------------------------------------
3.2.0 20600 ± 800 0.99849 ± 6e-05 1e-05 ± 1e-05 78 ± 4
3.3.0 100000 ± 3000 0.73 ± 0.05 4e-06 ± 3e-06 15.4 ± 0.5
3.4.1 100000 ± 5000 0.75 ± 0.04 1.5e-05 ± 3e-06 15.5 ± 0.7
dev 104000 ± 4000 0.71 ± 0.04 1.7e-05 ± 9e-06 15.0 ± 0.5
========================================================================
Wall time [sampling interval: 100]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
====================================================================
Sample Rate Saturation Error Rate Sampling Speed
--------------------------------------------------------------------
3.2.0 9000 ± 400 0.09 ± 0.01 2e-05 ± 3e-05 104 ± 5
3.3.0 10500 ± 300 0.0044 ± 0.0009 1e-05 ± 2e-05 18.4 ± 0.8
3.4.1 10200 ± 300 0.0042 ± 0.0007 5e-05 ± 6e-05 18.6 ± 0.5
dev 10400 ± 300 0.0042 ± 0.0009 6e-05 ± 6e-05 18.5 ± 0.7
====================================================================
Wall time [sampling interval: 1000]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
======================================================================
Sample Rate Saturation Error Rate Sampling Speed
----------------------------------------------------------------------
3.2.0 1800 ± 40 0.0009 ± 0.0009 3e-05 ± 9e-05 88 ± 3
3.3.0 1820 ± 40 0.0002 ± 0.0003 0 ± 0 25 ± 1
3.4.1 1810 ± 30 0.0005 ± 0.0005 0.0001 ± 0.0001 28 ± 2
dev 1810 ± 30 0.0006 ± 0.0005 3e-05 ± 8e-05 26 ± 2
======================================================================
CPU time [sampling interval: 1]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
================================================================
Sample Rate Saturation Error Rate Sampling Speed
----------------------------------------------------------------
3.2.0 16600 ± 700 1 ± 0 0 ± 0 76 ± 2
3.3.0 43000 ± 2000 1 ± 0 0 ± 0 27.0 ± 0.7
3.4.1 44000 ± 1000 1 ± 0 2e-05 ± 1e-05 27.1 ± 0.9
dev 44000 ± 2000 1 ± 0 3e-05 ± 2e-05 27 ± 1
================================================================
CPU time [sampling interval: 10]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
================================================================
Sample Rate Saturation Error Rate Sampling Speed
----------------------------------------------------------------
3.2.0 16700 ± 500 1 ± 0 0 ± 0 76 ± 2
3.3.0 43000 ± 3000 1 ± 0 0 ± 0 27 ± 1
3.4.1 44000 ± 1000 1 ± 0 2e-05 ± 1e-05 27.5 ± 0.8
dev 44000 ± 1000 1 ± 0 2e-05 ± 1e-05 27 ± 1
================================================================
CPU time [sampling interval: 100]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
=====================================================================
Sample Rate Saturation Error Rate Sampling Speed
---------------------------------------------------------------------
3.2.0 8000 ± 2000 0.28 ± 0.08 0 ± 0 96 ± 9
3.3.0 5000 ± 90 0.009 ± 0.003 0 ± 0 31 ± 2
3.4.1 4990 ± 90 0.008 ± 0.001 0.00017 ± 0.0001 32 ± 2
dev 5100 ± 200 0.007 ± 0.003 0.0001 ± 0.0001 30 ± 2
=====================================================================
CPU time [sampling interval: 1000]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
======================================================================
Sample Rate Saturation Error Rate Sampling Speed
----------------------------------------------------------------------
3.2.0 930 ± 4 0.002 ± 0.001 0 ± 0 80 ± 5
3.3.0 932 ± 4 0.0007 ± 0.0009 0 ± 0 41 ± 2
3.4.1 930 ± 2 0.0006 ± 0.0006 0.0002 ± 0.0004 42 ± 3
dev 930 ± 4 0.0005 ± 0.0006 0.0003 ± 0.0003 43 ± 3
======================================================================
RSA keygen [sampling interval: 1]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
==================================================================
Sample Rate Saturation Error Rate Sampling Speed
------------------------------------------------------------------
3.2.0 10000 ± 2000 1 ± 0 0 ± 0 110 ± 20
3.3.0 36000 ± 1000 1 ± 0 2e-05 ± 6e-05 27 ± 1
3.4.1 33000 ± 2000 1 ± 0 0.0002 ± 0.0001 30 ± 2
dev 34000 ± 2000 1 ± 0 0.0002 ± 0.0002 29 ± 2
==================================================================
RSA keygen [sampling interval: 10]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
=======================================================================
Sample Rate Saturation Error Rate Sampling Speed
-----------------------------------------------------------------------
3.2.0 9300 ± 200 1 ± 0 0 ± 0 107 ± 3
3.3.0 35000 ± 1000 1 ± 0 1e-05 ± 2e-05 28 ± 1
3.4.1 32000 ± 2000 1 ± 0 0.0003 ± 0.0002 30 ± 1
dev 33000 ± 3000 0.99999 ± 2e-05 0.0002 ± 0.0003 30 ± 2
=======================================================================
RSA keygen [sampling interval: 100]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
====================================================================
Sample Rate Saturation Error Rate Sampling Speed
--------------------------------------------------------------------
3.2.0 8500 ± 300 0.98 ± 0.01 0 ± 0 115 ± 4
3.3.0 6100 ± 100 0.004 ± 0.004 0 ± 0 31 ± 2
3.4.1 6100 ± 100 0.017 ± 0.01 0.0005 ± 0.0004 37 ± 4
dev 6070 ± 70 0.014 ± 0.009 0.0004 ± 0.0004 36 ± 3
====================================================================
RSA keygen [sampling interval: 1000]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
====================================================================
Sample Rate Saturation Error Rate Sampling Speed
--------------------------------------------------------------------
3.2.0 920 ± 10 0.0003 ± 0.0008 0 ± 0 130 ± 20
3.3.0 910 ± 50 0 ± 0 0 ± 0 48 ± 6
3.4.1 921 ± 8 0.001 ± 0.002 0.002 ± 0.002 57 ± 7
dev 925 ± 8 0.001 ± 0.002 0.0 ± 0.001 52 ± 7
====================================================================
Full metrics [sampling interval: 1]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
================================================================
Sample Rate Saturation Error Rate Sampling Speed
----------------------------------------------------------------
3.2.0 16300 ± 600 1 ± 0 0.0 ± 1e-05 98 ± 4
3.3.0 43000 ± 1000 1 ± 0 1e-06 ± 4e-06 36 ± 1
3.4.1 42000 ± 2000 1 ± 0 3e-05 ± 2e-05 36 ± 2
dev 43000 ± 2000 1 ± 0 4e-05 ± 2e-05 36 ± 1
================================================================
Full metrics [sampling interval: 10]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
================================================================
Sample Rate Saturation Error Rate Sampling Speed
----------------------------------------------------------------
3.2.0 16000 ± 1000 1 ± 0 1e-05 ± 1e-05 99 ± 7
3.3.0 42000 ± 2000 1 ± 0 0 ± 0 37 ± 2
3.4.1 43600 ± 1000 1 ± 0 3e-05 ± 1e-05 35.3 ± 0.8
dev 43000 ± 2000 1 ± 0 4e-05 ± 1e-05 36 ± 2
================================================================
Full metrics [sampling interval: 100]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
====================================================================
Sample Rate Saturation Error Rate Sampling Speed
--------------------------------------------------------------------
3.2.0 16100 ± 200 0.496 ± 0.005 0 ± 0 97 ± 1
3.3.0 9400 ± 400 0.016 ± 0.003 0 ± 0 45 ± 2
3.4.1 9400 ± 200 0.014 ± 0.002 0.00011 ± 6e-05 45 ± 1
dev 9100 ± 400 0.018 ± 0.003 0.0001 ± 5e-05 45 ± 2
====================================================================
Full metrics [sampling interval: 1000]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
======================================================================
Sample Rate Saturation Error Rate Sampling Speed
----------------------------------------------------------------------
3.2.0 1810 ± 30 0.0009 ± 0.0006 0 ± 0 115 ± 3
3.3.0 1780 ± 30 0.0003 ± 0.0004 0 ± 0 55 ± 1
3.4.1 1770 ± 30 0.0006 ± 0.0005 0.0001 ± 0.0001 56 ± 2
dev 1790 ± 30 0.001 ± 0.0009 0.0001 ± 0.0001 56 ± 3
======================================================================
Multiprocess wall time [sampling interval: 1]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
=================================================================
Sample Rate Saturation Error Rate Sampling Speed
-----------------------------------------------------------------
3.2.0 370 ± 20 1 ± 0 0.0002 ± 0.0001 1200 ± 40
3.3.0 920 ± 80 1 ± 0 0.0005 ± 0.0008 440 ± 40
3.4.1 1900 ± 200 1 ± 0 0.0004 ± 0.0005 330 ± 20
dev 2700 ± 100 1 ± 0 0.00018 ± 5e-05 350 ± 20
=================================================================
Multiprocess wall time [sampling interval: 10]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
=================================================================
Sample Rate Saturation Error Rate Sampling Speed
-----------------------------------------------------------------
3.2.0 350 ± 20 1 ± 0 0.00016 ± 6e-05 1250 ± 70
3.3.0 940 ± 30 1 ± 0 0.0003 ± 0.0006 420 ± 40
3.4.1 2000 ± 200 1 ± 0 0.0002 ± 0.0003 320 ± 20
dev 2800 ± 100 1 ± 0 0.00017 ± 7e-05 350 ± 10
=================================================================
Multiprocess wall time [sampling interval: 100]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
======================================================================
Sample Rate Saturation Error Rate Sampling Speed
----------------------------------------------------------------------
3.2.0 370 ± 30 0.9984 ± 0.0009 0.00015 ± 4e-05 1210 ± 60
3.3.0 940 ± 70 0.07 ± 0.02 4e-05 ± 5e-05 450 ± 20
3.4.1 1900 ± 100 0.04 ± 0.02 0.0002 ± 0.0003 330 ± 30
dev 2560 ± 90 0.05 ± 0.02 0.0001 ± 3e-05 370 ± 10
======================================================================
Multiprocess wall time [sampling interval: 1000]
> Running with Austin 3.2.0 ...
> Running with Austin 3.3.0 ...
> Running with Austin 3.4.1 ...
> Running with Austin dev ...
====================================================================
Sample Rate Saturation Error Rate Sampling Speed
--------------------------------------------------------------------
3.2.0 340 ± 20 0.13 ± 0.01 0.00016 ± 5e-05 1230 ± 70
3.3.0 890 ± 80 0.05 ± 0.03 3e-05 ± 4e-05 300 ± 200
3.4.1 1600 ± 100 0.005 ± 0.007 3e-05 ± 3e-05 70 ± 30
dev 2300 ± 200 0.02 ± 0.008 2e-05 ± 2e-05 140 ± 40
====================================================================
Cherry-pick of #177 to master.