search

P403n1x87 / austin

Python frame stack sampler for CPython
https://pypi.org/project/austin-dist/
GNU General Public License v3.0
1.89k stars 54 forks source link

chore(austinp): clean up eval frame detection #207

Closed P403n1x87 closed 1 year ago

codecov[bot] commented 1 year ago

Codecov Report

All modified lines are covered by tests :white_check_mark:

Comparison is base (3263bd7) 68.72% compared to head (e8fa0e0) 68.32%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## devel #207 +/- ## ========================================== - Coverage 68.72% 68.32% -0.41% ========================================== Files 27 27 Lines 2523 2519 -4 Branches 775 774 -1 ========================================== - Hits 1734 1721 -13 - Misses 457 466 +9 Partials 332 332 ``` | [Files](https://app.codecov.io/gh/P403n1x87/austin/pull/207?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gabriele+N.+Tornetta) | Coverage Δ | | |---|---|---| | [src/py\_thread.c](https://app.codecov.io/gh/P403n1x87/austin/pull/207?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gabriele+N.+Tornetta#diff-c3JjL3B5X3RocmVhZC5j) | `72.30% <100.00%> (+0.74%)` | :arrow_up: | ... and [4 files with indirect coverage changes](https://app.codecov.io/gh/P403n1x87/austin/pull/207/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gabriele+N.+Tornetta)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 1 year ago

Austin Benchmarks

Running Austin benchmarks with Python 3.10.13

Benchmark Summary

Comparison of dev against 3.5.0.

The following scenarios show a statistically significant difference in performance between the two versions.

Sample Rate Saturation Error Rate Sampling Speed
CPU time [sampling interval: 100] :yellow_circle: :yellow_circle: :green_circle: :yellow_circle:
Full metrics [sampling interval: 100] :yellow_circle: :yellow_circle: :green_circle: :yellow_circle:
Multiprocess wall time [sampling interval: 100] :yellow_circle: :yellow_circle: :green_circle: :yellow_circle:

Benchmark Results

Wall time [sampling interval: 1] ## Wall time [sampling interval: 1] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 118000 ± 4000 | 1 ± 0 | 7e-06 ± 4e-06 | 13.2 ± 0.4 | | dev | 121000 ± 3000 | 1 ± 0 | 7e-06 ± 3e-06 | 12.8 ± 0.4 |
Wall time [sampling interval: 10] ## Wall time [sampling interval: 10] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 117000 ± 7000 | 0.543 ± 0.001 | 7e-06 ± 4e-06 | 13.1 ± 0.9 | | dev | 118000 ± 4000 | 0.543 ± 0.002 | 7e-06 ± 3e-06 | 13.2 ± 0.4 |
Wall time [sampling interval: 100] ## Wall time [sampling interval: 100] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 10800 ± 500 | 0.0007 ± 0.0001 | 2e-05 ± 2e-05 | 14.4 ± 0.5 | | dev | 10600 ± 400 | 0.0006 ± 0.0002 | 1e-05 ± 2e-05 | 14.4 ± 0.5 |
Wall time [sampling interval: 1000] ## Wall time [sampling interval: 1000] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 1820 ± 40 | 0.0003 ± 0.0004 | 0.0001 ± 0.0001 | 18 ± 2 | | dev | 1850 ± 30 | 0.0002 ± 0.0003 | 0 ± 0 | 18 ± 1 |
CPU time [sampling interval: 1] ## CPU time [sampling interval: 1] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 49000 ± 3000 | 1 ± 0 | 1.6e-05 ± 5e-06 | 23.4 ± 1.0 | | dev | 51000 ± 4000 | 1 ± 0 | 1.4e-05 ± 9e-06 | 23 ± 1 |
CPU time [sampling interval: 10] ## CPU time [sampling interval: 10] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 51000 ± 4000 | 0.998 ± 0.0004 | 1e-05 ± 1e-05 | 23 ± 1 | | dev | 52000 ± 2000 | 0.9979 ± 0.0005 | 1.2e-05 ± 6e-06 | 22.8 ± 0.8 |
CPU time [sampling interval: 100] ## CPU time [sampling interval: 100] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 5380 ± 100 | 0.001 ± 0.0002 | 0.00013 ± 8e-05 | 23.1 ± 0.3 | | dev | 5500 ± 100 | 0.0009 ± 0.0002 | 4e-05 ± 6e-05 | 23.2 ± 0.4 |
CPU time [sampling interval: 1000] ## CPU time [sampling interval: 1000] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 945 ± 6 | 0.0002 ± 0.0006 | 0.0001 ± 0.0002 | 28 ± 3 | | dev | 944 ± 7 | 0.0001 ± 0.0003 | 0.0 ± 0.0001 | 27.7 ± 0.9 |
RSA keygen [sampling interval: 1] ## RSA keygen [sampling interval: 1] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 39000 ± 2000 | 1 ± 0 | 0.0002 ± 0.0001 | 25 ± 1 | | dev | 39400 ± 900 | 1 ± 0 | 0.0002 ± 0.0002 | 24.8 ± 0.6 |
RSA keygen [sampling interval: 10] ## RSA keygen [sampling interval: 10] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 38000 ± 2000 | 0.97 ± 0.02 | 0.0004 ± 0.0004 | 25.1 ± 1.0 | | dev | 39000 ± 1000 | 0.979 ± 0.009 | 0.0002 ± 0.0002 | 24.7 ± 0.9 |
RSA keygen [sampling interval: 100] ## RSA keygen [sampling interval: 100] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 6300 ± 20 | 0.002 ± 0.001 | 0.0006 ± 0.0006 | 29 ± 2 | | dev | 6300 ± 100 | 0.002 ± 0.002 | 0.0005 ± 0.0007 | 28 ± 2 |
RSA keygen [sampling interval: 1000] ## RSA keygen [sampling interval: 1000] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 940 ± 5 | 0 ± 0 | 0.001 ± 0.002 | 36 ± 2 | | dev | 930 ± 10 | 0 ± 0 | 0.002 ± 0.003 | 36 ± 3 |
Full metrics [sampling interval: 1] ## Full metrics [sampling interval: 1] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 51000 ± 3000 | 1 ± 0 | 1.6e-05 ± 6e-06 | 30 ± 2 | | dev | 51000 ± 2000 | 1 ± 0 | 1.5e-05 ± 9e-06 | 30 ± 2 |
Full metrics [sampling interval: 10] ## Full metrics [sampling interval: 10] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 51000 ± 2000 | 1 ± 0 | 2e-05 ± 1e-05 | 30 ± 1 | | dev | 50000 ± 2000 | 1 ± 0 | 1.6e-05 ± 9e-06 | 30 ± 1 |
Full metrics [sampling interval: 100] ## Full metrics [sampling interval: 100] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 9800 ± 200 | 0.0018 ± 0.0003 | 4e-05 ± 3e-05 | 33.7 ± 0.7 | | dev | 9800 ± 200 | 0.0016 ± 0.0002 | 1e-05 ± 1e-05 | 33.5 ± 0.7 |
Full metrics [sampling interval: 1000] ## Full metrics [sampling interval: 1000] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 1850 ± 40 | 0.0003 ± 0.0005 | 0.0 ± 0.0001 | 39 ± 2 | | dev | 1840 ± 40 | 0.0004 ± 0.0004 | 0.0001 ± 0.0002 | 39 ± 1 |
Multiprocess wall time [sampling interval: 1] ## Multiprocess wall time [sampling interval: 1] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 3100 ± 100 | 1 ± 0 | 0.00018 ± 4e-05 | 300 ± 10 | | dev | 3200 ± 100 | 1 ± 0 | 0.00014 ± 6e-05 | 300 ± 10 |
Multiprocess wall time [sampling interval: 10] ## Multiprocess wall time [sampling interval: 10] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 3140 ± 90 | 1 ± 0 | 0.00015 ± 5e-05 | 306 ± 9 | | dev | 3100 ± 100 | 1 ± 0 | 0.00012 ± 5e-05 | 310 ± 10 |
Multiprocess wall time [sampling interval: 100] ## Multiprocess wall time [sampling interval: 100] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 3100 ± 100 | 0.049 ± 0.009 | 0.00011 ± 5e-05 | 310 ± 10 | | dev | 3100 ± 200 | 0.05 ± 0.01 | 5e-05 ± 1e-05 | 300 ± 20 |
Multiprocess wall time [sampling interval: 1000] ## Multiprocess wall time [sampling interval: 1000] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.5.0 | 2780 ± 90 | 0.01 ± 0.001 | 3e-05 ± 3e-05 | 82 ± 6 | | dev | 2840 ± 80 | 0.009 ± 0.001 | 1e-05 ± 2e-05 | 77 ± 7 |