search

P403n1x87 / austin

Python frame stack sampler for CPython
https://pypi.org/project/austin-dist/
GNU General Public License v3.0
1.71k stars 51 forks source link

chore: drop support for old Python versions #186

Closed P403n1x87 closed 1 year ago

P403n1x87 commented 1 year ago

Description of the Change

Python < 3.8 is no longer maintained, so we drop support for it. Previous versions of Austin can still be used if needed.

Regressions

None expected

Verification Process

The existing test suite has been streamlined by removing the runs for the versions that are no longer supported. Existing tests are kept for all supported versions.

codecov[bot] commented 1 year ago

Codecov Report

Patch coverage: 53.33% and project coverage change: -0.30% :warning:

Comparison is base (89a1f48) 70.80% compared to head (0706d07) 70.50%.

Additional details and impacted files ```diff @@ Coverage Diff @@ ## devel #186 +/- ## ========================================== - Coverage 70.80% 70.50% -0.30% ========================================== Files 25 25 Lines 2473 2397 -76 Branches 730 709 -21 ========================================== - Hits 1751 1690 -61 Misses 388 388 + Partials 334 319 -15 ``` | [Files Changed](https://app.codecov.io/gh/P403n1x87/austin/pull/186?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gabriele+N.+Tornetta) | Coverage Δ | | |---|---|---| | [src/linux/py\_proc.h](https://app.codecov.io/gh/P403n1x87/austin/pull/186?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gabriele+N.+Tornetta#diff-c3JjL2xpbnV4L3B5X3Byb2MuaA==) | `58.96% <ø> (-0.31%)` | :arrow_down: | | [src/py\_proc.c](https://app.codecov.io/gh/P403n1x87/austin/pull/186?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gabriele+N.+Tornetta#diff-c3JjL3B5X3Byb2MuYw==) | `68.11% <ø> (-0.64%)` | :arrow_down: | | [src/version.h](https://app.codecov.io/gh/P403n1x87/austin/pull/186?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gabriele+N.+Tornetta#diff-c3JjL3ZlcnNpb24uaA==) | `57.89% <0.00%> (+0.31%)` | :arrow_up: | | [src/py\_string.h](https://app.codecov.io/gh/P403n1x87/austin/pull/186?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gabriele+N.+Tornetta#diff-c3JjL3B5X3N0cmluZy5o) | `72.54% <53.57%> (-4.38%)` | :arrow_down: | | [src/py\_thread.c](https://app.codecov.io/gh/P403n1x87/austin/pull/186?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gabriele+N.+Tornetta#diff-c3JjL3B5X3RocmVhZC5j) | `74.43% <100.00%> (-0.06%)` | :arrow_down: | ... and [1 file with indirect coverage changes](https://app.codecov.io/gh/P403n1x87/austin/pull/186/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=Gabriele+N.+Tornetta)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

github-actions[bot] commented 1 year ago

Austin Benchmarks

Running Austin benchmarks with Python 3.10.12

Wall time [sampling interval: 1] ## Wall time [sampling interval: 1] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 115000 ± 5000 | 1 ± 0 | 2e-05 ± 1e-05 | 13.6 ± 0.7 | | 3.5.0 | 116000 ± 5000 | 1 ± 0 | 1.9e-05 ± 7e-06 | 13.4 ± 0.5 | | dev | 120000 ± 3000 | 1 ± 0 | 1.7e-05 ± 6e-06 | 12.8 ± 0.4 |
Wall time [sampling interval: 10] ## Wall time [sampling interval: 10] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 115000 ± 7000 | 0.544 ± 0.002 | 1.3e-05 ± 6e-06 | 13.5 ± 0.8 | | 3.5.0 | 115000 ± 9000 | 0.545 ± 0.002 | 1.4e-05 ± 7e-06 | 14 ± 1 | | dev | 118000 ± 4000 | 0.545 ± 0.002 | 2.1e-05 ± 9e-06 | 13.1 ± 0.3 |
Wall time [sampling interval: 100] ## Wall time [sampling interval: 100] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 10200 ± 400 | 0.0007 ± 0.0002 | 6e-05 ± 8e-05 | 14.6 ± 0.5 | | 3.5.0 | 10500 ± 300 | 0.0007 ± 0.0002 | 6e-05 ± 5e-05 | 14.1 ± 0.3 | | dev | 10400 ± 200 | 0.0006 ± 0.0002 | 6e-05 ± 7e-05 | 14.5 ± 0.5 |
Wall time [sampling interval: 1000] ## Wall time [sampling interval: 1000] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 1790 ± 60 | 0 ± 0 | 0.0001 ± 0.0002 | 16.3 ± 0.5 | | 3.5.0 | 1810 ± 50 | 0 ± 0 | 0.0001 ± 0.0001 | 17.7 ± 0.5 | | dev | 1840 ± 60 | 0.0001 ± 0.0002 | 0.0001 ± 0.0002 | 17.8 ± 0.6 |
CPU time [sampling interval: 1] ## CPU time [sampling interval: 1] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 51000 ± 5000 | 1 ± 0 | 3e-05 ± 2e-05 | 23 ± 1 | | 3.5.0 | 52000 ± 3000 | 1 ± 0 | 2e-05 ± 1e-05 | 22.5 ± 0.7 | | dev | 51000 ± 6000 | 1 ± 0 | 3e-05 ± 2e-05 | 23 ± 1 |
CPU time [sampling interval: 10] ## CPU time [sampling interval: 10] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 52000 ± 3000 | 0.9967 ± 0.0003 | 2e-05 ± 2e-05 | 22.4 ± 0.5 | | 3.5.0 | 49000 ± 5000 | 0.9965 ± 0.0004 | 3e-05 ± 2e-05 | 23 ± 1 | | dev | 53000 ± 3000 | 0.9964 ± 0.0005 | 2.8e-05 ± 8e-06 | 22.5 ± 0.7 |
CPU time [sampling interval: 100] ## CPU time [sampling interval: 100] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 5400 ± 100 | 0.0013 ± 0.0003 | 0.0002 ± 0.0001 | 23 ± 1 | | 3.5.0 | 5400 ± 300 | 0.001 ± 0.001 | 0.00012 ± 9e-05 | 24 ± 4 | | dev | 5400 ± 300 | 0.0012 ± 0.0003 | 8e-05 ± 9e-05 | 22.5 ± 0.5 |
CPU time [sampling interval: 1000] ## CPU time [sampling interval: 1000] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 950 ± 4 | 0 ± 0 | 0.0001 ± 0.0003 | 26.6 ± 0.7 | | 3.5.0 | 950 ± 2 | 0 ± 0 | 0.0001 ± 0.0003 | 25.6 ± 0.5 | | dev | 951 ± 3 | 0.0001 ± 0.0002 | 0.0001 ± 0.0002 | 25.9 ± 0.6 |
RSA keygen [sampling interval: 1] ## RSA keygen [sampling interval: 1] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 40000 ± 1000 | 1 ± 0 | 0.0002 ± 0.0001 | 24.4 ± 0.7 | | 3.5.0 | 39500 ± 900 | 1 ± 0 | 0.0002 ± 0.0002 | 24.7 ± 0.5 | | dev | 40000 ± 1000 | 1 ± 0 | 0.0002 ± 0.0003 | 24.0 ± 0.7 |
RSA keygen [sampling interval: 10] ## RSA keygen [sampling interval: 10] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 39600 ± 400 | 0.984 ± 0.009 | 0.0002 ± 0.0001 | 24.4 ± 0.5 | | 3.5.0 | 39300 ± 500 | 0.976 ± 0.008 | 0.0003 ± 0.0002 | 24.7 ± 0.5 | | dev | 41000 ± 1000 | 0.97 ± 0.02 | 0.0003 ± 0.0002 | 23.8 ± 0.6 |
RSA keygen [sampling interval: 100] ## RSA keygen [sampling interval: 100] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 6260 ± 40 | 0.003 ± 0.002 | 0.0011 ± 0.0009 | 30 ± 3 | | 3.5.0 | 6240 ± 40 | 0.002 ± 0.002 | 0.001 ± 0.002 | 29 ± 1 | | dev | 6280 ± 40 | 0.0013 ± 0.0009 | 0.0004 ± 0.0004 | 28 ± 1 |
RSA keygen [sampling interval: 1000] ## RSA keygen [sampling interval: 1000] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 942 ± 2 | 0 ± 0 | 0.002 ± 0.002 | 34 ± 2 | | 3.5.0 | 943 ± 1 | 0 ± 0 | 0.001 ± 0.001 | 32 ± 2 | | dev | 942 ± 4 | 0 ± 0 | 0.001 ± 0.001 | 30 ± 1 |
Full metrics [sampling interval: 1] ## Full metrics [sampling interval: 1] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 51000 ± 2000 | 1 ± 0 | 5e-05 ± 1e-05 | 30 ± 1 | | 3.5.0 | 51000 ± 3000 | 1 ± 0 | 5e-05 ± 2e-05 | 30 ± 2 | | dev | 51000 ± 2000 | 1 ± 0 | 0.0001 ± 0.0002 | 30 ± 1 |
Full metrics [sampling interval: 10] ## Full metrics [sampling interval: 10] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 51000 ± 2000 | 1 ± 0 | 4e-05 ± 2e-05 | 30 ± 1 | | 3.5.0 | 51000 ± 2000 | 1 ± 0 | 5e-05 ± 2e-05 | 30 ± 1 | | dev | 51000 ± 2000 | 1 ± 0 | 4e-05 ± 2e-05 | 30 ± 2 |
Full metrics [sampling interval: 100] ## Full metrics [sampling interval: 100] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 10000 ± 400 | 0.0019 ± 0.0005 | 0.00013 ± 8e-05 | 32.4 ± 0.8 | | 3.5.0 | 10000 ± 200 | 0.0017 ± 0.0003 | 9e-05 ± 7e-05 | 32.4 ± 1.0 | | dev | 10000 ± 300 | 0.0018 ± 0.0003 | 6e-05 ± 6e-05 | 32.5 ± 0.7 |
Full metrics [sampling interval: 1000] ## Full metrics [sampling interval: 1000] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 1869 ± 5 | 0.0001 ± 0.0002 | 0 ± 0 | 36.5 ± 0.5 | | 3.5.0 | 1850 ± 20 | 0.0001 ± 0.0003 | 0.0 ± 0.0001 | 36.6 ± 1.0 | | dev | 1860 ± 20 | 0.0 ± 0.0001 | 0.0001 ± 0.0002 | 37.0 ± 0.7 |
Multiprocess wall time [sampling interval: 1] ## Multiprocess wall time [sampling interval: 1] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 3100 ± 200 | 1 ± 0 | 0.00018 ± 5e-05 | 310 ± 20 | | 3.5.0 | 3500 ± 200 | 1 ± 0 | 0.00016 ± 4e-05 | 280 ± 20 | | dev | 3500 ± 100 | 1 ± 0 | 0.00012 ± 2e-05 | 280 ± 10 |
Multiprocess wall time [sampling interval: 10] ## Multiprocess wall time [sampling interval: 10] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 3000 ± 200 | 1 ± 0 | 0.00013 ± 3e-05 | 310 ± 20 | | 3.5.0 | 3400 ± 200 | 1 ± 0 | 0.00014 ± 3e-05 | 290 ± 20 | | dev | 3500 ± 200 | 1 ± 0 | 0.00014 ± 2e-05 | 270 ± 20 |
Multiprocess wall time [sampling interval: 100] ## Multiprocess wall time [sampling interval: 100] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 3000 ± 200 | 0.051 ± 0.006 | 7e-05 ± 6e-05 | 310 ± 20 | | 3.5.0 | 3400 ± 200 | 0.04 ± 0.02 | 0.0001 ± 4e-05 | 280 ± 20 | | dev | 3300 ± 200 | 0.04 ± 0.02 | 8e-05 ± 4e-05 | 290 ± 20 |
Multiprocess wall time [sampling interval: 1000] ## Multiprocess wall time [sampling interval: 1000] | | Sample Rate | Saturation | Error Rate | Sampling Speed | | --- |:-----------:|:----------:|:----------:|:--------------:| | 3.4.1 | 2800 ± 100 | 0.012 ± 0.005 | 3e-05 ± 3e-05 | 90 ± 20 | | 3.5.0 | 3100 ± 100 | 0.004 ± 0.002 | 2e-05 ± 3e-05 | 53 ± 7 | | dev | 3090 ± 90 | 0.005 ± 0.002 | 2e-05 ± 2e-05 | 54 ± 8 |

Benchmark Summary

Comparison of dev against 3.5.0.

The following scenarios show a statistically significant difference in performance between the two versions.

Sample Rate Saturation Error Rate Sampling Speed
Wall time [sampling interval: 1] :green_circle: :yellow_circle: :yellow_circle: :green_circle:
CPU time [sampling interval: 10] :green_circle: :yellow_circle: :yellow_circle: :yellow_circle:
RSA keygen [sampling interval: 1] :yellow_circle: :yellow_circle: :yellow_circle: :green_circle:
RSA keygen [sampling interval: 10] :green_circle: :yellow_circle: :yellow_circle: :green_circle:
RSA keygen [sampling interval: 1000] :yellow_circle: :yellow_circle: :yellow_circle: :green_circle:
Multiprocess wall time [sampling interval: 1] :yellow_circle: :yellow_circle: :green_circle: :yellow_circle: