Open kloczek opened 1 year ago
Looks like this unit should be skipped when there is no git
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-benchmark-4.0.0-3.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-benchmark-4.0.0-3.fc35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra -m 'not network' --ignore tests/test_elasticsearch_storage.py ============================= test session starts ============================== platform linux -- Python 3.8.16, pytest-7.2.1, pluggy-1.0.0 benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000) rootdir: /home/tkloczko/rpmbuild/BUILD/pytest-benchmark-4.0.0, configfile: pytest.ini, testpaths: tests plugins: benchmark-4.0.0, xdist-3.2.0, aspectlib-1.5.2 collected 225 items tests/test_benchmark.py ................................................ [ 21%] ..... [ 23%] tests/test_calibration.py .................................. [ 38%] tests/test_cli.py ........... [ 43%] tests/test_doctest.rst . [ 44%] tests/test_normal.py .......... [ 48%] tests/test_pedantic.py ................... [ 56%] tests/test_sample.py .... [ 58%] tests/test_skip.py s [ 59%] tests/test_stats.py .............. [ 65%] tests/test_storage.py ........................................ [ 83%] tests/test_utils.py ...ssssssssss.F...ss.ss.ss.ss..... [ 98%] tests/test_with_testcase.py .. [ 99%] tests/test_with_weaver.py .. [100%] =================================== FAILURES =================================== ____________________________ test_commit_info_error ____________________________ /home/tkloczko/rpmbuild/BUILD/pytest-benchmark-4.0.0/tests/test_utils.py:127: in test_commit_info_error assert info['error'].lower().startswith("calledprocesserror(128, 'fatal: not a git repository") E assert False E + where False = <built-in method startswith of str object at 0x7fea33260dc0>("calledprocesserror(128, 'fatal: not a git repository") E + where <built-in method startswith of str object at 0x7fea33260dc0> = "filenotfounderror(2, 'no such file or directory')".startswith E + where "filenotfounderror(2, 'no such file or directory')" = <built-in method lower of str object at 0x7fea33260ea0>() E + where <built-in method lower of str object at 0x7fea33260ea0> = "FileNotFoundError(2, 'No such file or directory')".lower ----------------------------------------------------------------------------------------------------------------------------------- benchmark: 61 tests ------------------------------------------------------------------------------------------------------------------------------------ Name (time in ns) Min Max Mean StdDev Median IQR Outliers OPS Rounds Iterations -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- test_calibrate_stuck[False-0-1e-10] 5.0500 (1.0) 5.0500 (1.0) 5.0500 (1.0) 0.0000 (1.0) 5.0500 (1.0) 0.0000 (1.0) 0;0 198,019,801.9802 (1.0) 1 2 test_calibrate_stuck[False-1-1e-10] 5.0500 (1.0) 5.0500 (1.0) 5.0500 (1.0) 0.0000 (1.0) 5.0500 (1.0) 0.0000 (1.0) 0;0 198,019,801.9802 (1.0) 1 2 test_calibrate_stuck[False--1-1e-10] 5.0500 (1.00) 5.0500 (1.00) 5.0500 (1.00) 0.0000 (1.0) 5.0500 (1.00) 0.0000 (1.0) 0;0 198,019,801.9796 (1.00) 1 2 test_calibrate_stuck[True--1-1e-10] 10.0000 (1.98) 10.0000 (1.98) 10.0000 (1.98) 0.0000 (1.0) 10.0000 (1.98) 0.0000 (1.0) 0;0 100,000,000.0001 (0.51) 1 1 test_calibrate_stuck[True-0-1e-10] 10.0000 (1.98) 10.0000 (1.98) 10.0000 (1.98) 0.0000 (1.0) 10.0000 (1.98) 0.0000 (1.0) 0;0 100,000,000.0001 (0.51) 1 1 test_calibrate_stuck[True-1-1e-10] 10.0000 (1.98) 10.0000 (1.98) 10.0000 (1.98) 0.0000 (1.0) 10.0000 (1.98) 0.0000 (1.0) 0;0 100,000,000.0001 (0.51) 1 1 test_calibrate_stuck[False--1-1e-09] 50.5000 (10.00) 50.5000 (10.00) 50.5000 (10.00) 0.0000 (1.0) 50.5000 (10.00) 0.0000 (1.0) 0;0 19,801,980.1980 (0.10) 1 2 test_calibrate_stuck[False-0-1e-09] 50.5000 (10.00) 50.5000 (10.00) 50.5000 (10.00) 0.0000 (1.0) 50.5000 (10.00) 0.0000 (1.0) 0;0 19,801,980.1980 (0.10) 1 2 test_calibrate_stuck[False-1-1e-09] 50.5000 (10.00) 50.5000 (10.00) 50.5000 (10.00) 0.0000 (1.0) 50.5000 (10.00) 0.0000 (1.0) 0;0 19,801,980.1980 (0.10) 1 2 test_xfast 66.7572 (13.22) 123.9777 (24.55) 71.5150 (14.16) 3.5820 (inf) 71.5256 (14.16) 0.0000 (1.0) 22;375 13,983,078.7807 (0.07) 1354 100 test_calibrate_stuck[True--1-1e-09] 100.0000 (19.80) 100.0000 (19.80) 100.0000 (19.80) 0.0000 (1.0) 100.0000 (19.80) 0.0000 (1.0) 0;0 10,000,000.0000 (0.05) 1 1 test_calibrate_stuck[True-0-1e-09] 100.0000 (19.80) 100.0000 (19.80) 100.0000 (19.80) 0.0000 (1.0) 100.0000 (19.80) 0.0000 (1.0) 0;0 10,000,000.0000 (0.05) 1 1 test_calibrate_stuck[True-1-1e-09] 100.0000 (19.80) 100.0000 (19.80) 100.0000 (19.80) 0.0000 (1.0) 100.0000 (19.80) 0.0000 (1.0) 0;0 10,000,000.0000 (0.05) 1 1 test_calibrate_xfast 109.3117 (21.65) 6,494.5500 (>1000.0) 111.7397 (22.13) 18.7764 (inf) 110.8716 (21.95) 0.6286 (inf) 11454;118971 8,949,373.6255 (0.05) 941693 100 test_rounds_iterations 334.0188 (66.14) 560.8890 (111.07) 385.4496 (76.33) 62.9072 (inf) 362.0051 (71.68) 51.9562 (inf) 2;2 2,594,373.0546 (0.01) 15 10 test_proto[LocalsSimpleProxy] 402.7970 (79.76) 114,749.8027 (>1000.0) 456.2419 (90.34) 415.6832 (inf) 419.2465 (83.02) 33.2948 (inf) 236;12179 2,191,819.5905 (0.01) 78156 20 test_calibrate_fast 444.0080 (87.92) 53,213.6764 (>1000.0) 463.9979 (91.88) 96.5740 (inf) 460.5778 (91.20) 9.2356 (inf) 12260;66573 2,155,182.1205 (0.01) 1902533 12 test_rounds 447.9662 (88.71) 1,393.0257 (275.85) 638.6079 (126.46) 260.5217 (inf) 532.0180 (105.35) 276.4282 (inf) 2;1 1,565,906.1164 (0.01) 15 1 test_proto[LocalsCachedPropertyProxy] 448.8975 (88.89) 16,906.9972 (>1000.0) 511.4434 (101.28) 211.2767 (inf) 490.8070 (97.19) 28.1725 (inf) 1889;5077 1,955,250.6569 (0.01) 175105 1 test_warmup_rounds 523.8689 (103.74) 896.1651 (177.46) 634.4169 (125.63) 153.4711 (inf) 579.9811 (114.85) 174.0991 (inf) 1;0 1,576,250.4756 (0.01) 5 1 test_iterations 558.0019 (110.50) 558.0019 (110.50) 558.0019 (110.50) 0.0000 (1.0) 558.0019 (110.50) 0.0000 (1.0) 0;0 1,792,108.5271 (0.01) 1 10 test_proto[CachedPropertyProxy] 631.9024 (125.13) 9,700.8888 (>1000.0) 688.6240 (136.36) 126.2901 (inf) 679.1670 (134.49) 27.0084 (inf) 94;1542 1,452,171.2937 (0.01) 80127 1 test_proto[SimpleProxy] 738.0731 (146.15) 54,013.2169 (>1000.0) 800.9582 (158.61) 201.2293 (inf) 786.0363 (155.65) 21.1876 (inf) 2018;12366 1,248,504.6308 (0.01) 177456 1 test_setup_many_rounds 905.0127 (179.21) 2,499.9026 (495.03) 1,212.3259 (240.06) 479.6699 (inf) 1,027.9473 (203.55) 152.9697 (inf) 1;2 824,860.7225 (0.00) 10 1 test_single 1,452.8632 (287.70) 1,452.8632 (287.70) 1,452.8632 (287.70) 0.0000 (1.0) 1,452.8632 (287.70) 0.0000 (1.0) 0;0 688,296.0410 (0.00) 1 1 test_can_use_both_args_and_setup_without_return 1,922.0170 (380.60) 1,922.0170 (380.60) 1,922.0170 (380.60) 0.0000 (1.0) 1,922.0170 (380.60) 0.0000 (1.0) 0;0 520,286.7712 (0.00) 1 1 test_args_kwargs 3,504.1012 (693.88) 3,504.1012 (693.88) 3,504.1012 (693.88) 0.0000 (1.0) 3,504.1012 (693.88) 0.0000 (1.0) 0;0 285,379.8868 (0.00) 1 1 test_setup 3,807.0139 (753.86) 3,807.0139 (753.86) 3,807.0139 (753.86) 0.0000 (1.0) 3,807.0139 (753.86) 0.0000 (1.0) 0;0 262,673.0656 (0.00) 1 1 test_foo 6,034.9703 (>1000.0) 152,045.8609 (>1000.0) 64,790.4150 (>1000.0) 8,395.6336 (inf) 67,616.1144 (>1000.0) 147.8475 (inf) 2351;3620 15,434.3818 (0.00) 16167 1 test_fast 8,741.1609 (>1000.0) 97,583.9794 (>1000.0) 62,108.6773 (>1000.0) 8,485.5536 (inf) 60,020.9460 (>1000.0) 7,230.3228 (inf) 1462;1017 16,100.8098 (0.00) 14085 1 test_parametrized[0] 14,543.9990 (>1000.0) 115,676.0845 (>1000.0) 68,715.2769 (>1000.0) 9,043.4774 (inf) 64,658.0011 (>1000.0) 11,797.0631 (inf) 905;352 14,552.8046 (0.00) 14271 1 test_parametrized[1] 15,130.9650 (>1000.0) 148,372.9575 (>1000.0) 75,416.8173 (>1000.0) 6,040.9576 (inf) 76,527.0088 (>1000.0) 147.1490 (inf) 2827;4163 13,259.6420 (0.00) 14358 1 test_foo2 15,847.1521 (>1000.0) 78,969.1694 (>1000.0) 65,483.4025 (>1000.0) 8,188.2788 (inf) 67,656.8598 (>1000.0) 128.0569 (inf) 846;1687 15,271.0452 (0.00) 11936 1 test_setup_cprofile 18,751.0159 (>1000.0) 18,751.0159 (>1000.0) 18,751.0159 (>1000.0) 0.0000 (1.0) 18,751.0159 (>1000.0) 0.0000 (1.0) 0;0 53,330.4439 (0.00) 1 1 test_calibrate_slow 26,900.0884 (>1000.0) 3,155,903.8907 (>1000.0) 77,413.4750 (>1000.0) 9,333.6087 (inf) 76,854.1358 (>1000.0) 157.8592 (inf) 3542;106115 12,917.6477 (0.00) 683390 1 test_calibrate 44,321.8742 (>1000.0) 4,201,730.9461 (>1000.0) 46,529.1507 (>1000.0) 38,899.3567 (inf) 45,694.1780 (>1000.0) 604.8940 (inf) 155;23645 21,491.9031 (0.00) 225556 1 test_parametrized[4] 47,705.1362 (>1000.0) 138,234.8128 (>1000.0) 77,325.6275 (>1000.0) 2,465.4677 (inf) 76,652.0388 (>1000.0) 128.0569 (inf) 880;1719 12,932.3231 (0.00) 10735 1 test_parametrized[3] 54,944.0738 (>1000.0) 3,035,787.0273 (>1000.0) 77,622.7805 (>1000.0) 30,397.4406 (inf) 76,571.9451 (>1000.0) 117.8123 (inf) 3;1585 12,882.8160 (0.00) 9539 1 test_parametrized[2] 57,581.8121 (>1000.0) 122,495.9269 (>1000.0) 77,343.8228 (>1000.0) 2,520.5551 (inf) 76,618.9769 (>1000.0) 115.9497 (inf) 890;1732 12,929.2808 (0.00) 10802 1 test_slow 1,048,668.0549 (>1000.0) 1,193,783.9445 (>1000.0) 1,121,650.8574 (>1000.0) 43,953.8094 (inf) 1,095,492.8584 (>1000.0) 88,272.7909 (inf) 493;0 891.5430 (0.00) 929 1 test_slower 10,149,935.0276 (>1000.0) 10,202,737.0594 (>1000.0) 10,174,600.1088 (>1000.0) 10,092.0557 (inf) 10,174,617.1713 (>1000.0) 12,523.4947 (inf) 28;3 98.2840 (0.00) 99 1 test_weave_fixture 10,153,203.9698 (>1000.0) 10,199,252.9817 (>1000.0) 10,176,762.1934 (>1000.0) 18,238.2979 (inf) 10,170,449.9684 (>1000.0) 26,611.0292 (inf) 2;0 98.2631 (0.00) 5 1 test_weave_method 10,166,110.0052 (>1000.0) 10,181,545.0471 (>1000.0) 10,174,346.4824 (>1000.0) 7,295.0436 (inf) 10,177,953.1687 (>1000.0) 13,160.2283 (inf) 2;0 98.2864 (0.00) 5 1 test_calibrate_stuck[False--1-0.01] 504,999,999.9992 (>1000.0) 504,999,999.9992 (>1000.0) 504,999,999.9992 (>1000.0) 0.0000 (1.0) 504,999,999.9992 (>1000.0) 0.0000 (1.0) 0;0 1.9802 (0.00) 1 2 test_calibrate_stuck[False-0-0.01] 504,999,999.9992 (>1000.0) 504,999,999.9992 (>1000.0) 504,999,999.9992 (>1000.0) 0.0000 (1.0) 504,999,999.9992 (>1000.0) 0.0000 (1.0) 0;0 1.9802 (0.00) 1 2 test_calibrate_stuck[False-1-0.01] 504,999,999.9992 (>1000.0) 504,999,999.9992 (>1000.0) 504,999,999.9992 (>1000.0) 0.0000 (1.0) 504,999,999.9992 (>1000.0) 0.0000 (1.0) 0;0 1.9802 (0.00) 1 2 test_calibrate_stuck[True--1-0.01] 1,000,000,000.0000 (>1000.0) 1,000,000,000.0000 (>1000.0) 1,000,000,000.0000 (>1000.0) 0.0000 (1.0) 1,000,000,000.0000 (>1000.0) 0.0000 (1.0) 0;0 1.0000 (0.00) 1 1 test_calibrate_stuck[True-0-0.01] 1,000,000,000.0000 (>1000.0) 1,000,000,000.0000 (>1000.0) 1,000,000,000.0000 (>1000.0) 0.0000 (1.0) 1,000,000,000.0000 (>1000.0) 0.0000 (1.0) 0;0 1.0000 (0.00) 1 1 test_calibrate_stuck[True-1-0.01] 1,000,000,000.0000 (>1000.0) 1,000,000,000.0000 (>1000.0) 1,000,000,000.0000 (>1000.0) 0.0000 (1.0) 1,000,000,000.0000 (>1000.0) 0.0000 (1.0) 0;0 1.0000 (0.00) 1 1 test_calibrate_stuck[False--1-1.000000000000001] 50,500,000,000.0000 (>1000.0) 50,500,000,000.0000 (>1000.0) 50,500,000,000.0000 (>1000.0) 0.0000 (1.0) 50,500,000,000.0000 (>1000.0) 0.0000 (1.0) 0;0 0.0198 (0.00) 1 2 test_calibrate_stuck[False--1-1] 50,500,000,000.0000 (>1000.0) 50,500,000,000.0000 (>1000.0) 50,500,000,000.0000 (>1000.0) 0.0000 (1.0) 50,500,000,000.0000 (>1000.0) 0.0000 (1.0) 0;0 0.0198 (0.00) 1 2 test_calibrate_stuck[False-0-1.000000000000001] 50,500,000,000.0000 (>1000.0) 50,500,000,000.0000 (>1000.0) 50,500,000,000.0000 (>1000.0) 0.0000 (1.0) 50,500,000,000.0000 (>1000.0) 0.0000 (1.0) 0;0 0.0198 (0.00) 1 2 test_calibrate_stuck[False-0-1] 50,500,000,000.0000 (>1000.0) 50,500,000,000.0000 (>1000.0) 50,500,000,000.0000 (>1000.0) 0.0000 (1.0) 50,500,000,000.0000 (>1000.0) 0.0000 (1.0) 0;0 0.0198 (0.00) 1 2 test_calibrate_stuck[False-1-1.000000000000001] 50,500,000,000.0000 (>1000.0) 50,500,000,000.0000 (>1000.0) 50,500,000,000.0000 (>1000.0) 0.0000 (1.0) 50,500,000,000.0000 (>1000.0) 0.0000 (1.0) 0;0 0.0198 (0.00) 1 2 test_calibrate_stuck[False-1-1] 50,500,000,000.0000 (>1000.0) 50,500,000,000.0000 (>1000.0) 50,500,000,000.0000 (>1000.0) 0.0000 (1.0) 50,500,000,000.0000 (>1000.0) 0.0000 (1.0) 0;0 0.0198 (0.00) 1 2 test_calibrate_stuck[True--1-1.000000000000001] 100,000,000,000.0000 (>1000.0) 100,000,000,000.0000 (>1000.0) 100,000,000,000.0000 (>1000.0) 0.0000 (1.0) 100,000,000,000.0000 (>1000.0) 0.0000 (1.0) 0;0 0.0100 (0.00) 1 1 test_calibrate_stuck[True--1-1] 100,000,000,000.0000 (>1000.0) 100,000,000,000.0000 (>1000.0) 100,000,000,000.0000 (>1000.0) 0.0000 (1.0) 100,000,000,000.0000 (>1000.0) 0.0000 (1.0) 0;0 0.0100 (0.00) 1 1 test_calibrate_stuck[True-0-1.000000000000001] 100,000,000,000.0000 (>1000.0) 100,000,000,000.0000 (>1000.0) 100,000,000,000.0000 (>1000.0) 0.0000 (1.0) 100,000,000,000.0000 (>1000.0) 0.0000 (1.0) 0;0 0.0100 (0.00) 1 1 test_calibrate_stuck[True-0-1] 100,000,000,000.0000 (>1000.0) 100,000,000,000.0000 (>1000.0) 100,000,000,000.0000 (>1000.0) 0.0000 (1.0) 100,000,000,000.0000 (>1000.0) 0.0000 (1.0) 0;0 0.0100 (0.00) 1 1 test_calibrate_stuck[True-1-1.000000000000001] 100,000,000,000.0000 (>1000.0) 100,000,000,000.0000 (>1000.0) 100,000,000,000.0000 (>1000.0) 0.0000 (1.0) 100,000,000,000.0000 (>1000.0) 0.0000 (1.0) 0;0 0.0100 (0.00) 1 1 test_calibrate_stuck[True-1-1] 100,000,000,000.0000 (>1000.0) 100,000,000,000.0000 (>1000.0) 100,000,000,000.0000 (>1000.0) 0.0000 (1.0) 100,000,000,000.0000 (>1000.0) 0.0000 (1.0) 0;0 0.0100 (0.00) 1 1 -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Legend: Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile. OPS: Operations Per Second, computed as 1 / Mean ----------------------------- cProfile (time in s) ----------------------------- tests/test_pedantic.py::test_setup_cprofile ncalls tottime percall cumtime percall filename:lineno(function) 1 0.0000 0.0000 0.0000 0.0000 pytest-benchmark-4.0.0/tests/test_pedantic.py:29(stuff) 1 0.0000 0.0000 0.0000 0.0000 ~:0(<method 'append' of 'list' objects>) 1 0.0000 0.0000 0.0000 0.0000 ~:0(<method 'disable' of '_lsprof.Profiler' objects>) =========================== short test summary info ============================ SKIPPED [1] tests/test_skip.py:5: bla SKIPPED [2] tests/test_utils.py:60: 'git' not available on $PATH SKIPPED [2] tests/test_utils.py:60: 'hg' not available on $PATH SKIPPED [2] tests/test_utils.py:80: 'git' not available on $PATH SKIPPED [2] tests/test_utils.py:80: 'hg' not available on $PATH SKIPPED [1] tests/test_utils.py:94: 'git' not available on $PATH SKIPPED [1] tests/test_utils.py:94: 'hg' not available on $PATH SKIPPED [4] tests/test_utils.py:160: 'git' not available on $PATH SKIPPED [4] tests/test_utils.py:160: 'hg' not available on $PATH FAILED tests/test_utils.py::test_commit_info_error - assert False ============ 1 failed, 205 passed, 19 skipped in 478.31s (0:07:58) =============
Looks like this unit should be skipped when there is no git