search

microsoft / CCF

Confidential Consortium Framework
https://microsoft.github.io/CCF/
Apache License 2.0
766 stars 207 forks source link

Plot commits and posts in the analyzer #4563

Closed fotiskoun closed 1 year ago

fotiskoun commented 1 year ago

This comes as an addition to the analyzer component, implementing the a plot of commits and posts. In order for this plot to produce correct results, it is dependent, for now, on the structures of the requests produced from the generator component. So, the generator should produce requests like: N posts 1 commit and in the end end add multiple commits in order to get the final commit stage. This PR also removes some figure settings on the analyzer plots and two prints that are no longer needed.

achamayou commented 1 year ago

/azp run

azure-pipelines[bot] commented 1 year ago
Azure Pipelines successfully started running 2 pipeline(s).
ccf-bot commented 1 year ago

commit-plot-for-ccf-analyzer@56889 aka 20221201.18 vs main ewma over 20 builds from 56076 to 56881

Click to see table main | build_id | build_number | Commit latency factor | tpcc_virtual_cft^ | ls_virtual_cft^ | tpcc_sgx_cft^ | tpcc_sgx_cft_mem | ls_sgx_cft^ | ls_sgx_cft_mem | pi_ls_virtual_cft^ | ls_jwt_virtual_cft^ | ls_js_virtual_cft^ | ls_full_js_virtual_cft^ | pi_ls_sgx_cft^ | pi_ls_sgx_cft_mem | ls_js_jwt_virtual_cft^ | ls_jwt_sgx_cft^ | ls_jwt_sgx_cft_mem | ls_js_sgx_cft^ | ls_js_sgx_cft_mem | hist_sgx_cft^ | ls_full_js_sgx_cft^ | ls_full_js_sgx_cft_mem | ls_js_jwt_sgx_cft^ | ls_js_jwt_sgx_cft_mem | RB put (/s)^ | CHAMP put (/s)^ | RB get (/s)^ | CHAMP get (/s)^ | pi_ls_jwt_virtual_cft^ | pi_ls_jwt_sgx_cft^ | pi_ls_jwt_sgx_cft_mem | |-----------:|:---------------|------------------------:|--------------------:|------------------:|----------------:|-------------------:|--------------:|-----------------:|---------------------:|----------------------:|---------------------:|--------------------------:|-----------------:|--------------------:|-------------------------:|------------------:|---------------------:|-----------------:|--------------------:|----------------:|----------------------:|-------------------------:|---------------------:|------------------------:|---------------:|------------------:|---------------:|------------------:|-------------------------:|---------------------:|------------------------:| | 56076 | 20221125.64 | 0.778898 | 17660 | 43983.5 | 6373.41 | 8.27106e+07 | 15497.4 | 1.56017e+07 | 46522 | 12656.8 | 4683.76 | 3644.7 | 15568.1 | 8.52384e+06 | 3476.71 | 5822.16 | 1.50774e+07 | 1573.64 | 7.99955e+06 | 23161 | 1278.51 | 7.47526e+06 | 1300.94 | 6.95098e+06 | 835307 | 1.18442e+06 | 8.15455e+06 | 3.13966e+07 | nan | nan | nan | | 56085 | 20221128.2 | 0.75605 | 16824.2 | 45788.6 | 6304.21 | 8.21863e+07 | 15516.1 | 1.53396e+07 | 43343.7 | 12728.4 | 4515.21 | 3592.84 | 15576.9 | 8.78598e+06 | 3348.48 | 5661.29 | 1.50774e+07 | 1572.38 | 7.99955e+06 | 27115.5 | 1287.12 | 7.47526e+06 | 1303.02 | 7.21312e+06 | 840999 | 1.18059e+06 | 8.15559e+06 | 3.09207e+07 | nan | nan | nan | | 56212 | 20221128.35 | 0.784032 | 17609.2 | 43760.4 | 6433.11 | 8.24484e+07 | 15516.7 | 1.50774e+07 | 45736.3 | 12498.9 | 4602.15 | 3681.51 | 15555.9 | 8.78598e+06 | 3388.78 | 5863.29 | 1.50774e+07 | 1566.3 | 7.73741e+06 | 24063.5 | 1287.41 | 7.99955e+06 | 1301.48 | 6.95098e+06 | 836523 | 1.1821e+06 | 8.15462e+06 | 3.07341e+07 | nan | nan | nan | | 56236 | 20221128.41 | 0.768172 | 17383 | 45833.3 | 6361.44 | 8.24484e+07 | 15461.7 | 1.53396e+07 | 47312.5 | 12591 | 4443.63 | 3626.18 | 15519.2 | 8.78598e+06 | 3456.6 | 5835.73 | 1.53396e+07 | 1555.8 | 7.99955e+06 | 32006.2 | 1285.76 | 7.73741e+06 | 1308.51 | 6.95098e+06 | 840524 | 1.17456e+06 | 8.15543e+06 | 3.08304e+07 | nan | nan | nan | | 56328 | 20221128.68 | 0.766841 | 17199.9 | 47950.7 | 6327.82 | 8.21863e+07 | 15477.2 | 1.58639e+07 | 46153.3 | 12390 | 4692.09 | 3662.61 | 15567.1 | 8.52384e+06 | 3408.99 | 5702.1 | 1.50774e+07 | 1563.1 | 7.99955e+06 | 23711.9 | 1286.4 | 7.21312e+06 | 1304.93 | 6.95098e+06 | 832006 | 1.17709e+06 | 8.04162e+06 | 3.05075e+07 | nan | nan | nan | | 56335 | 20221128.70 | 0.789409 | 17658.6 | 45990.3 | 6311.67 | 8.24484e+07 | 15522.9 | 1.53396e+07 | 47174.3 | 12471.3 | 4536.31 | 3578.51 | 15577.5 | 8.78598e+06 | 3461.63 | 5838.17 | 1.50774e+07 | 1569.06 | 7.99955e+06 | 28381.9 | 1294.16 | 7.47526e+06 | 1307.92 | 7.21312e+06 | 839260 | 1.17907e+06 | 8.15491e+06 | 3.10115e+07 | nan | nan | nan | | 56342 | 20221128.72 | 0.79034 | 17051.7 | 48012.4 | 6392.43 | 8.24484e+07 | 15502 | 1.53396e+07 | 45150.2 | 12567.2 | 4657.64 | 3622.31 | 15586.6 | 8.52384e+06 | 3439.75 | 5843.36 | 1.53396e+07 | 1566.21 | 7.99955e+06 | 24048.8 | 1289.15 | 7.47526e+06 | 1303.99 | 7.21312e+06 | 842330 | 1.17711e+06 | 8.1518e+06 | 3.05758e+07 | nan | nan | nan | | 56399 | 20221129.4 | 0.75541 | 17053.9 | 43931.9 | 6333.29 | 8.27106e+07 | 15551.5 | 1.53396e+07 | 43948.4 | 12651.3 | 4367.63 | 3610.4 | 15584.4 | 8.52384e+06 | 3422.18 | 5812.73 | 1.50774e+07 | 1568.92 | 7.99955e+06 | 22418.8 | 1293.58 | 7.47526e+06 | 1307.71 | 6.95098e+06 | 837641 | 1.17994e+06 | 8.15309e+06 | 3.16362e+07 | nan | nan | nan | | 56446 | 20221129.17 | 0.786514 | 17519.6 | 45948.1 | 6263.03 | 8.24484e+07 | 15558 | 1.50774e+07 | 46433.3 | 12500.7 | 4613.72 | 3500.42 | 15573.8 | 8.78598e+06 | 3333.33 | 5592.11 | 1.48153e+07 | 1567.08 | 7.73741e+06 | 21552 | 1287.26 | 7.47526e+06 | 1291.44 | 7.21312e+06 | 830763 | 1.17598e+06 | 8.17258e+06 | 3.09076e+07 | nan | nan | nan | | 56667 | 20221130.3 | 0.766543 | 17231.4 | 43775.5 | 6310.89 | 8.27106e+07 | 15587.6 | 1.50774e+07 | 47083 | 12283.2 | 4428.53 | 3683.76 | 15546.9 | 8.78598e+06 | 3469.28 | 5781.86 | 1.45532e+07 | 1566.39 | 7.99955e+06 | 25075.5 | 1286.5 | 7.47526e+06 | 1307.46 | 7.21312e+06 | 827206 | 1.17455e+06 | 8.16799e+06 | 3.09787e+07 | nan | nan | nan | | 56669 | 20221130.4 | 0.763369 | 17311.6 | 44003.5 | 6288.61 | 8.21863e+07 | 15441.5 | 1.50774e+07 | 46471 | 12499.7 | 4728.18 | 3514.69 | 15518.9 | 8.78598e+06 | 3458.6 | 5784.43 | 1.48153e+07 | 1571.22 | 7.99955e+06 | 28972.8 | 1288.54 | 7.47526e+06 | 1296.89 | 7.21312e+06 | 830809 | 1.17931e+06 | 8.17346e+06 | 3.16969e+07 | nan | nan | nan | | 56721 | 20221130.17 | 0.79414 | 17120.1 | 43773.4 | 6372.24 | 8.24484e+07 | 15543.7 | 1.53396e+07 | 44424.4 | 12473.4 | 4538.86 | 3576.6 | 15501 | 8.78598e+06 | 3439.4 | 5781.59 | 1.50774e+07 | 1570.62 | 7.99955e+06 | 25382.7 | 1288.25 | 7.21312e+06 | 1303.01 | 6.95098e+06 | 835926 | 1.1802e+06 | 8.15306e+06 | 3.07739e+07 | nan | nan | nan | | 56750 | 20221130.25 | 0.768941 | 17250.7 | 43871.5 | 6281.87 | 8.24484e+07 | 15336.9 | 1.50774e+07 | 45136.5 | 12529.7 | 4542.48 | 3643.52 | 15425.6 | 8.52384e+06 | 3422.02 | 5812.06 | 1.50774e+07 | 1539.43 | 7.99955e+06 | 24664.1 | 1284.03 | 7.47526e+06 | 1303.38 | 6.95098e+06 | 831768 | 1.17963e+06 | 8.15442e+06 | 3.07512e+07 | nan | nan | nan | | 56769 | 20221130.30 | 0.781149 | 17197.4 | 43850 | 6255.83 | 8.24484e+07 | 15359.2 | 1.53396e+07 | 44762.3 | 12446.2 | 4529.64 | 3673.19 | 15473.3 | 8.52384e+06 | 3481.86 | 5846.3 | 1.48153e+07 | 1539.42 | 7.99955e+06 | 22905.7 | 1282.47 | 7.73741e+06 | 1287.77 | 6.95098e+06 | 826101 | 1.17629e+06 | 8.15118e+06 | 3.11488e+07 | nan | nan | nan | | 56805 | 20221130.40 | 0.749085 | 16977.4 | 43514.9 | 6253.03 | 8.21863e+07 | 15306.1 | 1.50774e+07 | 46485.7 | 12323.4 | 4716.21 | 3521.35 | 15508.4 | 8.78598e+06 | 3411.84 | 5855.83 | 1.50774e+07 | 1561.44 | 7.99955e+06 | 27015.9 | 1282.81 | 7.21312e+06 | 1289.55 | 6.95098e+06 | 840034 | 1.17617e+06 | 8.15517e+06 | 3.12352e+07 | nan | nan | nan | | 56823 | 20221130.45 | 0.773479 | 17779.8 | 48236.3 | 6287.47 | 8.27106e+07 | 15337.6 | 1.50774e+07 | 47114.8 | 12547.6 | 4482.6 | 3700.46 | 15518.4 | 8.52384e+06 | 3433.77 | 5787.58 | 1.56017e+07 | 1561.86 | 7.99955e+06 | 23157.2 | 1285.24 | 7.21312e+06 | 1295.16 | 7.21312e+06 | 841024 | 1.18268e+06 | 8.15264e+06 | 3.07808e+07 | nan | nan | nan | | 56840 | 20221201.2 | 0.775331 | 17239.8 | 43871.8 | 6323.75 | 8.19242e+07 | 15528.1 | 1.50774e+07 | 46253.4 | 12484.7 | 4447.87 | 3605.37 | 15577.6 | 8.52384e+06 | 3505.57 | 5866 | 1.50774e+07 | 1566.75 | 7.99955e+06 | 23990.1 | 1288.71 | 7.47526e+06 | 1305.62 | 7.21312e+06 | 832802 | 1.1803e+06 | 8.15628e+06 | 3.21972e+07 | nan | nan | nan | | 56849 | 20221201.7 | 0.773313 | 17213.2 | 45830 | 6245.05 | 8.24484e+07 | 15409.6 | 1.58639e+07 | 45009.7 | 12463.5 | 4389.29 | 3519.59 | 15497.7 | 8.78598e+06 | 3301.84 | 5750.77 | 1.50774e+07 | 1564.48 | 7.99955e+06 | 22973.2 | 1293.92 | 7.73741e+06 | 1304.67 | 7.21312e+06 | 838482 | 1.17823e+06 | 8.15124e+06 | 3.06743e+07 | 13247.3 | 5834.7 | 7.21312e+06 | | 56869 | 20221201.13 | 0.77135 | 17485.3 | 43971.2 | 6352.17 | 8.27106e+07 | 15423.6 | 1.50774e+07 | 47123.6 | 12462.8 | 4542.03 | 3670.85 | 15510.9 | 8.78598e+06 | 3273.02 | 5828.73 | 1.53396e+07 | 1569.14 | 7.99955e+06 | 26858.5 | 1288.76 | 7.47526e+06 | 1296.45 | 7.21312e+06 | 841455 | 1.17639e+06 | 8.15387e+06 | 3.07411e+07 | 13319.3 | 5831.3 | 7.73741e+06 | | 56881 | 20221201.17 | 0.783562 | 17563.7 | 43781.1 | 6269.71 | 8.19242e+07 | 15310.3 | 1.50774e+07 | 46945.6 | 12571.2 | 4412.03 | 3627.46 | 15459.4 | 8.78598e+06 | 3518.19 | 5652.1 | 1.50774e+07 | 1559.84 | 7.99955e+06 | 26150.1 | 1285.47 | 7.21312e+06 | 1305.76 | 7.21312e+06 | 836372 | 1.17896e+06 | 8.17418e+06 | 3.12309e+07 | 14093 | 5806.7 | 7.21312e+06 | commit-plot-for-ccf-analyzer | build_id | build_number | Commit latency factor | tpcc_sgx_cft^ | tpcc_sgx_cft_mem | tpcc_virtual_cft^ | ls_sgx_cft^ | ls_sgx_cft_mem | ls_virtual_cft^ | pi_ls_sgx_cft^ | pi_ls_sgx_cft_mem | pi_ls_virtual_cft^ | ls_jwt_sgx_cft^ | ls_jwt_sgx_cft_mem | ls_jwt_virtual_cft^ | ls_js_virtual_cft^ | ls_js_sgx_cft^ | ls_js_sgx_cft_mem | ls_full_js_virtual_cft^ | ls_js_jwt_virtual_cft^ | ls_full_js_sgx_cft^ | ls_full_js_sgx_cft_mem | hist_sgx_cft^ | ls_js_jwt_sgx_cft^ | ls_js_jwt_sgx_cft_mem | RB put (/s)^ | CHAMP put (/s)^ | RB get (/s)^ | CHAMP get (/s)^ | pi_ls_jwt_virtual_cft^ | pi_ls_jwt_sgx_cft^ | pi_ls_jwt_sgx_cft_mem | |-----------:|:---------------|------------------------:|----------------:|-------------------:|--------------------:|--------------:|-----------------:|------------------:|-----------------:|--------------------:|---------------------:|------------------:|---------------------:|----------------------:|---------------------:|-----------------:|--------------------:|--------------------------:|-------------------------:|----------------------:|-------------------------:|----------------:|---------------------:|------------------------:|---------------:|------------------:|---------------:|------------------:|-------------------------:|---------------------:|------------------------:| | 56195 | 20221128.29 | 0.777425 | 6268.61 | 8.21863e+07 | 17451.1 | 15582.5 | 1.53396e+07 | 46071.3 | 15624.2 | 8.78598e+06 | 44256.3 | 5826.26 | 1.45532e+07 | 12566.8 | 4499.64 | 1569.77 | 7.99955e+06 | 3691.88 | 3425.79 | 1288.06 | 7.47526e+06 | 23251.8 | 1293.1 | 6.95098e+06 | 836124 | 1.17888e+06 | 8.14988e+06 | 3.13289e+07 | nan | nan | nan | | 56217 | 20221128.36 | 0.760683 | 6342.01 | 8.24484e+07 | 17751 | 15411.2 | 1.53396e+07 | 45794.1 | 15537.8 | 8.52384e+06 | 44974.6 | 5892.55 | 1.53396e+07 | 12630.4 | 4592.16 | 1565.16 | 7.99955e+06 | 3530.65 | 3473.07 | 1279.99 | 7.73741e+06 | 28816.2 | 1291.81 | 6.95098e+06 | 833891 | 1.17891e+06 | 8.15556e+06 | 3.07337e+07 | nan | nan | nan | | 56844 | 20221201.4 | 0.766338 | 6295.9 | 8.24484e+07 | 17621.2 | 15436.9 | 1.53396e+07 | 46164.3 | 15497.3 | 8.52384e+06 | 45081.4 | 5675.24 | 1.45532e+07 | 12516.5 | 4539.86 | 1562.98 | 7.99955e+06 | 3674.9 | 3388.33 | 1286.53 | 7.21312e+06 | 24699.7 | 1295.82 | 7.21312e+06 | 836721 | 1.17348e+06 | 8.16496e+06 | 3.00218e+07 | nan | nan | nan | | 56868 | 20221201.12 | 0.80364 | 6334.29 | 8.27106e+07 | 17265.7 | 15504.9 | 1.50774e+07 | 43846.2 | 15596 | 8.78598e+06 | 46354.1 | 5659.89 | 1.50774e+07 | 12483.6 | 4672.24 | 1566.3 | 7.99955e+06 | 3477.5 | 3419.52 | 1285.38 | 7.47526e+06 | 23761.5 | 1306.25 | 7.21312e+06 | 835499 | 1.18099e+06 | 8.12605e+06 | 3.08229e+07 | 13115.4 | 5828 | 7.21312e+06 | | 56889 | 20221201.18 | 0.77817 | 6318.91 | 8.24484e+07 | 17632.7 | 15413.1 | 1.50774e+07 | 43722.6 | 15528.2 | 8.78598e+06 | 47055 | 5849.79 | 1.56017e+07 | 12565.3 | 4358.77 | 1565.23 | 7.99955e+06 | 3513.75 | 3340.11 | 1291.09 | 7.21312e+06 | 24607.6 | 1293.6 | 7.21312e+06 | 839226 | 1.18084e+06 | 8.13955e+06 | 3.07914e+07 | 14068.4 | 5768 | 7.73741e+06 |

images

achamayou commented 1 year ago

/azp run

azure-pipelines[bot] commented 1 year ago
Azure Pipelines successfully started running 2 pipeline(s).
fotiskoun commented 1 year ago

@microsoft-github-policy-service agree

achamayou commented 1 year ago

/azp run

azure-pipelines[bot] commented 1 year ago
Azure Pipelines successfully started running 2 pipeline(s).
achamayou commented 1 year ago

/azp run

azure-pipelines[bot] commented 1 year ago
Azure Pipelines successfully started running 2 pipeline(s).
achamayou commented 1 year ago

/azp run

azure-pipelines[bot] commented 1 year ago
Azure Pipelines successfully started running 2 pipeline(s).
achamayou commented 1 year ago

/azp run

azure-pipelines[bot] commented 1 year ago
Azure Pipelines successfully started running 2 pipeline(s).
achamayou commented 1 year ago

/azp run

azure-pipelines[bot] commented 1 year ago
Azure Pipelines successfully started running 2 pipeline(s).