search

P403n1x87 / austin

Python frame stack sampler for CPython
https://pypi.org/project/austin-dist/
GNU General Public License v3.0
1.71k stars 51 forks source link

fix(austinp): ensure threads are resumed on sampling errors #199

Closed P403n1x87 closed 1 year ago

P403n1x87 commented 1 year ago

We make sure that threads are resumed regardless of potential errors during the sampling of the threads.

Requirements for Adding, Changing, Fixing or Removing a Feature

Fill out the template below. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion.

Description of the Change

Alternate Designs

Regressions

Verification Process

codecov[bot] commented 1 year ago

Codecov Report

Patch coverage: 78.43% and project coverage change: -0.01% :warning:

Comparison is base (aa557cb) 67.98% compared to head (5964c1b) 67.98%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## devel #199 +/- ## ========================================== - Coverage 67.98% 67.98% -0.01% ========================================== Files 27 27 Lines 2499 2508 +9 Branches 765 766 +1 ========================================== + Hits 1699 1705 +6 - Misses 464 465 +1 - Partials 336 338 +2 ``` | [Files Changed](https://app.codecov.io/gh/P403n1x87/austin/pull/199?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gabriele+N.+Tornetta) | Coverage Δ | | |---|---|---| | [src/py\_proc.c](https://app.codecov.io/gh/P403n1x87/austin/pull/199?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gabriele+N.+Tornetta#diff-c3JjL3B5X3Byb2MuYw==) | `62.85% <78.43%> (+0.08%)` | :arrow_up: | ... and [4 files with indirect coverage changes](https://app.codecov.io/gh/P403n1x87/austin/pull/199/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gabriele+N.+Tornetta)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 1 year ago

Austin Benchmarks

Running Austin benchmarks with Python 3.10.13

Benchmark Summary

Comparison of dev against 3.5.0.

The following scenarios show a statistically significant difference in performance between the two versions.

Sample Rate Saturation Error Rate Sampling Speed
Wall time [sampling interval: 1] :green_circle: :yellow_circle: :yellow_circle: :yellow_circle:
CPU time [sampling interval: 1] :green_circle: :yellow_circle: :yellow_circle: :yellow_circle:
CPU time [sampling interval: 100] :yellow_circle: :yellow_circle: :green_circle: :yellow_circle:

Benchmark Results

Wall time [sampling interval: 1] ## Wall time [sampling interval: 1] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 118000 ± 2000 | 1 ± 0 | 8e-06 ± 4e-06 | 13.2 ± 0.4 | | 3.5.0 | 119000 ± 3000 | 1 ± 0 | 9e-06 ± 3e-06 | 13.1 ± 0.6 | | dev | 121000 ± 2000 | 1 ± 0 | 8e-06 ± 4e-06 | 12.9 ± 0.3 |
Wall time [sampling interval: 10] ## Wall time [sampling interval: 10] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 118000 ± 3000 | 0.5443 ± 0.0009 | 6e-06 ± 4e-06 | 13.2 ± 0.4 | | 3.5.0 | 118000 ± 4000 | 0.544 ± 0.001 | 8e-06 ± 4e-06 | 13.1 ± 0.3 | | dev | 120000 ± 3000 | 0.544 ± 0.002 | 7e-06 ± 6e-06 | 13.1 ± 0.3 |
Wall time [sampling interval: 100] ## Wall time [sampling interval: 100] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 10300 ± 300 | 0.0008 ± 0.0001 | 3e-05 ± 2e-05 | 15.6 ± 0.5 | | 3.5.0 | 10400 ± 300 | 0.0008 ± 0.0001 | 2e-05 ± 3e-05 | 15.7 ± 0.8 | | dev | 10300 ± 200 | 0.0008 ± 0.0002 | 2e-05 ± 3e-05 | 15.1 ± 0.7 |
Wall time [sampling interval: 1000] ## Wall time [sampling interval: 1000] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 1860 ± 20 | 0.0002 ± 0.0003 | 0.0001 ± 0.0001 | 21 ± 1 | | 3.5.0 | 1860 ± 20 | 0.0001 ± 0.0002 | 2e-05 ± 6e-05 | 20.4 ± 1.0 | | dev | 1850 ± 20 | 0.0001 ± 0.0002 | 4e-05 ± 8e-05 | 21.1 ± 1.0 |
CPU time [sampling interval: 1] ## CPU time [sampling interval: 1] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 52000 ± 2000 | 1 ± 0 | 1.3e-05 ± 6e-06 | 23.0 ± 0.7 | | 3.5.0 | 52000 ± 1000 | 1 ± 0 | 1e-05 ± 9e-06 | 22.8 ± 0.4 | | dev | 53000 ± 1000 | 1 ± 0 | 1e-05 ± 5e-06 | 22.4 ± 0.5 |
CPU time [sampling interval: 10] ## CPU time [sampling interval: 10] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 52000 ± 1000 | 0.9983 ± 0.0002 | 1.8e-05 ± 9e-06 | 22.9 ± 0.3 | | 3.5.0 | 52000 ± 2000 | 0.9984 ± 0.0003 | 1.4e-05 ± 5e-06 | 22.9 ± 0.6 | | dev | 52000 ± 2000 | 0.998 ± 0.001 | 1.2e-05 ± 6e-06 | 22.8 ± 0.4 |
CPU time [sampling interval: 100] ## CPU time [sampling interval: 100] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 5300 ± 100 | 0.0015 ± 0.0005 | 6e-05 ± 4e-05 | 25 ± 1 | | 3.5.0 | 5250 ± 50 | 0.0014 ± 0.0003 | 0.00013 ± 5e-05 | 24.7 ± 0.7 | | dev | 5400 ± 300 | 0.0015 ± 0.0004 | 6e-05 ± 8e-05 | 24.7 ± 0.9 |
CPU time [sampling interval: 1000] ## CPU time [sampling interval: 1000] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 946 ± 2 | 0.0005 ± 0.0005 | 0.0001 ± 0.0002 | 34 ± 2 | | 3.5.0 | 946 ± 3 | 0.0005 ± 0.0009 | 0.0001 ± 0.0002 | 35 ± 4 | | dev | 947 ± 7 | 0.0004 ± 0.0005 | 0.0001 ± 0.0002 | 35 ± 2 |
RSA keygen [sampling interval: 1] ## RSA keygen [sampling interval: 1] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 39000 ± 1000 | 1 ± 0 | 0.0002 ± 0.0002 | 24.8 ± 0.8 | | 3.5.0 | 39200 ± 800 | 1 ± 0 | 0.0002 ± 0.0002 | 24.9 ± 0.3 | | dev | 39000 ± 1000 | 1 ± 0 | 0.0002 ± 0.0002 | 25.1 ± 0.7 |
RSA keygen [sampling interval: 10] ## RSA keygen [sampling interval: 10] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 38000 ± 3000 | 0.983 ± 0.009 | 0.0003 ± 0.0002 | 26 ± 2 | | 3.5.0 | 39000 ± 1000 | 0.986 ± 0.009 | 0.0003 ± 0.0003 | 24.8 ± 0.8 | | dev | 39400 ± 600 | 0.98 ± 0.01 | 0.0002 ± 0.0002 | 24.7 ± 0.7 |
RSA keygen [sampling interval: 100] ## RSA keygen [sampling interval: 100] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 6290 ± 20 | 0.003 ± 0.002 | 0.0007 ± 0.0005 | 28 ± 1 | | 3.5.0 | 6290 ± 10 | 0.003 ± 0.002 | 0.0005 ± 0.0005 | 28 ± 1 | | dev | 6260 ± 50 | 0.004 ± 0.004 | 0.001 ± 0.001 | 30 ± 2 |
RSA keygen [sampling interval: 1000] ## RSA keygen [sampling interval: 1000] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 938 ± 6 | 0 ± 0 | 0.001 ± 0.003 | 36 ± 3 | | 3.5.0 | 940 ± 2 | 0 ± 0 | 0.001 ± 0.002 | 36 ± 4 | | dev | 937 ± 7 | 0.001 ± 0.002 | 0.002 ± 0.002 | 38 ± 6 |
Full metrics [sampling interval: 1] ## Full metrics [sampling interval: 1] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 51000 ± 2000 | 1 ± 0 | 0.0001 ± 0.0001 | 30 ± 1 | | 3.5.0 | 51200 ± 800 | 1 ± 0 | 2.3e-05 ± 9e-06 | 30.1 ± 0.6 | | dev | 51000 ± 2000 | 1 ± 0 | 2.1e-05 ± 8e-06 | 30 ± 1 |
Full metrics [sampling interval: 10] ## Full metrics [sampling interval: 10] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 51000 ± 1000 | 1 ± 0 | 2.5e-05 ± 1e-05 | 30.2 ± 0.8 | | 3.5.0 | 51000 ± 1000 | 1 ± 0 | 2.4e-05 ± 7e-06 | 30 ± 1 | | dev | 51000 ± 2000 | 1 ± 0 | 3e-05 ± 1e-05 | 30 ± 1 |
Full metrics [sampling interval: 100] ## Full metrics [sampling interval: 100] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 9800 ± 300 | 0.0024 ± 0.0005 | 5e-05 ± 4e-05 | 35.2 ± 0.9 | | 3.5.0 | 9800 ± 300 | 0.0024 ± 0.0004 | 4e-05 ± 3e-05 | 35.3 ± 0.7 | | dev | 9700 ± 200 | 0.0023 ± 0.0004 | 3e-05 ± 5e-05 | 34.7 ± 0.9 |
Full metrics [sampling interval: 1000] ## Full metrics [sampling interval: 1000] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 1850 ± 30 | 0.0003 ± 0.0003 | 2e-05 ± 6e-05 | 41.1 ± 1.0 | | 3.5.0 | 1850 ± 20 | 0.0004 ± 0.0004 | 0 ± 0 | 41 ± 1 | | dev | 1860 ± 20 | 0.0003 ± 0.0004 | 0.0001 ± 0.0001 | 41 ± 2 |
Multiprocess wall time [sampling interval: 1] ## Multiprocess wall time [sampling interval: 1] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 2500 ± 200 | 1 ± 0 | 0.00018 ± 5e-05 | 257 ± 10 | | 3.5.0 | 3000 ± 100 | 1 ± 0 | 0.0003 ± 0.0003 | 320 ± 10 | | dev | 3000 ± 100 | 1 ± 0 | 0.00017 ± 7e-05 | 320 ± 10 |
Multiprocess wall time [sampling interval: 10] ## Multiprocess wall time [sampling interval: 10] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 2400 ± 200 | 1 ± 0 | 0.0003 ± 0.0004 | 270 ± 20 | | 3.5.0 | 3000 ± 200 | 1 ± 0 | 0.0002 ± 0.0003 | 320 ± 20 | | dev | 3200 ± 200 | 1 ± 0 | 0.00015 ± 4e-05 | 300 ± 20 |
Multiprocess wall time [sampling interval: 100] ## Multiprocess wall time [sampling interval: 100] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 2300 ± 100 | 0.03 ± 0.02 | 0.00011 ± 4e-05 | 270 ± 20 | | 3.5.0 | 3000 ± 200 | 0.05 ± 0.02 | 9e-05 ± 3e-05 | 320 ± 20 | | dev | 3000 ± 200 | 0.04 ± 0.02 | 9e-05 ± 5e-05 | 310 ± 20 |
Multiprocess wall time [sampling interval: 1000] ## Multiprocess wall time [sampling interval: 1000] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 2100 ± 200 | 0.001 ± 0.0004 | 4e-05 ± 4e-05 | 45 ± 1 | | 3.5.0 | 2900 ± 100 | 0.012 ± 0.003 | 3e-05 ± 2e-05 | 90 ± 10 | | dev | 2900 ± 100 | 0.011 ± 0.002 | 1e-05 ± 2e-05 | 86 ± 8 |