search

kruize / autotune-results

Recommendations and Results from Autotune
Apache License 2.0
3 stars 6 forks source link

Upload data for tpeMultiVariate algo for cpu and mem request for petclinic benchmark #8

Closed agarwalrounak closed 3 years ago

agarwalrounak commented 3 years ago

Closes #3

Signed-off-by: Rounak Agarwal rounag16@in.ibm.com

agarwalrounak commented 3 years ago
CPU_REQ MEM_REQ CPU_LIM MEM_LIM
4.19 687.53M 4.19 687.53M

./run-petclinic-openshift.sh -s towelled.os.fyre.ibm.com -e result -u 400 -d 60 -w 5 -m 3 -i 1 --iter=2 -r --cpulim=4.19 --cpureq=4.19 --memlim=687.53M --memreq=687.53M

Instances , Throughput , Responsetime , MEM_MEAN , CPU_MEAN , CPU_MIN , CPU_MAX , MEM_MIN , MEM_MAX , CLUSTER_MEM% , CLUSTER_CPU% , CPU_REQ , MEM_REQ , CPU_LIM , MEM_LIM , WEB_ERRORS 
1 ,  281.817 , 992.833 , 339.849 , 0.932864 , .7363174264323209 , 1.3141649768771444 , 329 , 349 , 17.4404 , 12.0238 , 4.19 , 687.53M , 4.19 , 687.53M , 0.166667
Run , CPU_REQ , MEM_REQ , CPU_LIM , MEM_LIM , Throughput , Responsetime , WEB_ERRORS , CPU , CPU_MIN , CPU_MAX , MEM , MEM_MIN , MEM_MAX
0 , 4.19 , 687.53M , 4.19 , 687.53M , 282.9 , 840 , 0   ,.906437 , .806977482146925 , 1.1258287749327425    , 348.306 , 348 , 348 
1 , 4.19 , 687.53M , 4.19 , 687.53M , 304.9 , 860 , 0   ,.872423 , .8096334503233238 , .950083345823626 , 348.899 , 348 , 349 
2 , 4.19 , 687.53M , 4.19 , 687.53M , 309.2 , 840 , 0   ,.850334 , .8193544229924998 , 1.0757903751970892   , 348.625 , 348 , 348 
0 , 4.19 , 687.53M , 4.19 , 687.53M , 282.9 , 840 , 0   ,.906437 , .806977482146925 , 1.1258287749327425    , 348.306 , 348 , 348 
1 , 4.19 , 687.53M , 4.19 , 687.53M , 304.9 , 860 , 0   ,.872423 , .8096334503233238 , .950083345823626 , 348.899 , 348 , 349 
2 , 4.19 , 687.53M , 4.19 , 687.53M , 309.2 , 840 , 0   ,.850334 , .8193544229924998 , 1.0757903751970892   , 348.625 , 348 , 348 
0 , 4.19 , 687.53M , 4.19 , 687.53M , 322.8 , 796 , 0   ,1.04254 , .887249742840302 , 1.3141649768771444    , 330.101 , 329 , 330 
1 , 4.19 , 687.53M , 4.19 , 687.53M , 165.3 , 1768 , 1  ,1.0699 , .949648184549626 , 1.2594402249668462 , 331.207 , 330 , 331 
2 , 4.19 , 687.53M , 4.19 , 687.53M , 305.8 , 853 , 0   ,.85555 , .7363174264323209 , 1.0669671577154243    , 331.954 , 331 , 332
agarwalrounak commented 3 years ago
CPU_REQ MEM_REQ CPU_LIM MEM_LIM
4.19 625.52M 4.19 625.52M

./run-petclinic-openshift.sh -s towelled.os.fyre.ibm.com -e result -u 400 -d 60 -w 5 -m 3 -i 1 --iter=2 -r --cpulim=4.19 --cpureq=4.19 --memlim=625.52M --memreq=625.52M

Instances , Throughput , Responsetime , MEM_MEAN , CPU_MEAN , CPU_MIN , CPU_MAX , MEM_MIN , MEM_MAX , CLUSTER_MEM% , CLUSTER_CPU% , CPU_REQ , MEM_REQ , CPU_LIM , MEM_LIM , WEB_ERRORS 
1 ,  265.683 , 1171.83 , 326.568 , 0.855323 , .6075867917828024 , 1.0883765412329396 , 322 , 331 , 17.1166 , 11.4159 , 4.19 , 625.52M , 4.19 , 625.52M , 0.5
Run , CPU_REQ , MEM_REQ , CPU_LIM , MEM_LIM , Throughput , Responsetime , WEB_ERRORS , CPU , CPU_MIN , CPU_MAX , MEM , MEM_MIN , MEM_MAX
0 , 4.19 , 625.52M , 4.19 , 625.52M , 304.2 , 860 , 0   ,.847206 , .7292658739247916 , 1.031601214304655    , 329.262 , 329 , 329 
1 , 4.19 , 625.52M , 4.19 , 625.52M , 301.7 , 868 , 0   ,.853009 , .7160422126612174 , 1.0191491503936543   , 330.318 , 330 , 330 
2 , 4.19 , 625.52M , 4.19 , 625.52M , 304.2 , 860 , 0   ,.863786 , .7595720030603054 , .9620232718381863    , 330.696 , 330 , 331 
0 , 4.19 , 625.52M , 4.19 , 625.52M , 304.2 , 860 , 0   ,.847206 , .7292658739247916 , 1.031601214304655    , 329.262 , 329 , 329 
1 , 4.19 , 625.52M , 4.19 , 625.52M , 301.7 , 868 , 0   ,.853009 , .7160422126612174 , 1.0191491503936543   , 330.318 , 330 , 330 
2 , 4.19 , 625.52M , 4.19 , 625.52M , 304.2 , 860 , 0   ,.863786 , .7595720030603054 , .9620232718381863    , 330.696 , 330 , 331 
0 , 4.19 , 625.52M , 4.19 , 625.52M , 310.7 , 837 , 0   ,.917691 , .8378990470220004 , 1.0343495157959506   , 322.099 , 322 , 322 
1 , 4.19 , 625.52M , 4.19 , 625.52M , 64.7 , 2763 , 3   ,.955353 , .8239459864119486 , 1.0883765412329396   , 323.904 , 322 , 330 
2 , 4.19 , 625.52M , 4.19 , 625.52M , 308.6 , 843 , 0   ,.694893 , .6075867917828024 , .8658577095601692    , 323.129 , 323 , 323
agarwalrounak commented 3 years ago
CPU_REQ MEM_REQ CPU_LIM MEM_LIM
4.45 622.68M 4.45 622.68M

./run-petclinic-openshift.sh -s towelled.os.fyre.ibm.com -e result -u 400 -d 60 -w 5 -m 3 -i 1 --iter=2 -r --cpulim=4.45 --cpureq=4.45 --memlim=622.68M --memreq=622.68M

Instances , Throughput , Responsetime , MEM_MEAN , CPU_MEAN , CPU_MIN , CPU_MAX , MEM_MIN , MEM_MAX , CLUSTER_MEM% , CLUSTER_CPU% , CPU_REQ , MEM_REQ , CPU_LIM , MEM_LIM , WEB_ERRORS 
1 ,  275.817 , 864.333 , 343.916 , 0.885766 , .7472515525643343 , 1.1927665394479927 , 325 , 363 , 17.1786 , 11.9567 , 4.45 , 622.68M , 4.45 , 622.68M , 0
Run , CPU_REQ , MEM_REQ , CPU_LIM , MEM_LIM , Throughput , Responsetime , WEB_ERRORS , CPU , CPU_MIN , CPU_MAX , MEM , MEM_MIN , MEM_MAX
0 , 4.45 , 622.68M , 4.45 , 622.68M , 138.7 , 944 , 0   ,.844863 , .7472515525643343 , 1.0514346379595807   , 360.418 , 360 , 360 
1 , 4.45 , 622.68M , 4.45 , 622.68M , 292.0 , 864 , 0   ,.824562 , .7586414733822386 , 1.0352817953539861   , 361.476 , 361 , 361 
2 , 4.45 , 622.68M , 4.45 , 622.68M , 305.5 , 855 , 0   ,.831863 , .7502589745969072 , 1.010689466931718    , 362.348 , 361 , 363 
0 , 4.45 , 622.68M , 4.45 , 622.68M , 138.7 , 944 , 0   ,.844863 , .7472515525643343 , 1.0514346379595807   , 360.418 , 360 , 360 
1 , 4.45 , 622.68M , 4.45 , 622.68M , 292.0 , 864 , 0   ,.824562 , .7586414733822386 , 1.0352817953539861   , 361.476 , 361 , 361 
2 , 4.45 , 622.68M , 4.45 , 622.68M , 305.5 , 855 , 0   ,.831863 , .7502589745969072 , 1.010689466931718    , 362.348 , 361 , 363 
0 , 4.45 , 622.68M , 4.45 , 622.68M , 310.1 , 839 , 0   ,.95935 , .7960366851085022 , 1.1927665394479927    , 325.775 , 325 , 326 
1 , 4.45 , 622.68M , 4.45 , 622.68M , 309.6 , 840 , 0   ,.919739 , .8325319109597932 , 1.0767059781439279   , 326.564 , 326 , 326 
2 , 4.45 , 622.68M , 4.45 , 622.68M , 299.0 , 844 , 0   ,.934218 , .8096274937668797 , 1.1678838191310026   , 326.913 , 326 , 327
agarwalrounak commented 3 years ago

No requests, no limits

./run-petclinic-openshift.sh -s towelled.os.fyre.ibm.com -e result -u 400 -d 60 -w 5 -m 3 -i 1 --iter=2 -r

Instances , Throughput , Responsetime , MEM_MEAN , CPU_MEAN , CPU_MIN , CPU_MAX , MEM_MIN , MEM_MAX , CLUSTER_MEM% , CLUSTER_CPU% , CPU_REQ , MEM_REQ , CPU_LIM , MEM_LIM , WEB_ERRORS 
1 ,  312.583 , 835.5 , 502.835 , 0.853224 , .5732975631168286 , 1.266957914343397 , 487 , 518 , 17.6051 , 12.6668 ,  ,  ,  ,  , 0
Run , CPU_REQ , MEM_REQ , CPU_LIM , MEM_LIM , Throughput , Responsetime , WEB_ERRORS , CPU , CPU_MIN , CPU_MAX , MEM , MEM_MIN , MEM_MAX
0 ,  ,  ,  ,  , 324.5 , 789 , 0 ,.9622 , .8599369698599999 , 1.1506228723996945 , 517.341 , 517 , 517 
1 ,  ,  ,  ,  , 332.3 , 768 , 0 ,1.00079 , .8946479308479155 , 1.12362754889976 , 517.658 , 517 , 517 
2 ,  ,  ,  ,  , 340.0 , 743 , 0 ,1.05391 , .9373663963414636 , 1.266957914343397    , 518.382 , 518 , 518 
0 ,  ,  ,  ,  , 324.5 , 789 , 0 ,.9622 , .8599369698599999 , 1.1506228723996945 , 517.341 , 517 , 517 
1 ,  ,  ,  ,  , 332.3 , 768 , 0 ,1.00079 , .8946479308479155 , 1.12362754889976 , 517.658 , 517 , 517 
2 ,  ,  ,  ,  , 340.0 , 743 , 0 ,1.05391 , .9373663963414636 , 1.266957914343397    , 518.382 , 518 , 518 
0 ,  ,  ,  ,  , 269.0 , 1000 , 0    ,.692924 , .5961489552410225 , .9301842361154932    , 488.102 , 487 , 488 
1 ,  ,  ,  ,  , 304.9 , 857 , 0 ,.653577 , .5732975631168286 , .8401437234608203    , 487.508 , 487 , 487 
2 ,  ,  ,  ,  , 304.8 , 856 , 0 ,.755945 , .7046439830209633 , .8463729947151506    , 488.02 , 487 , 488
agarwalrounak commented 3 years ago

No requests, max limits

./run-petclinic-openshift.sh -s towelled.os.fyre.ibm.com -e result -u 400 -d 60 -w 5 -m 3 -i 1 --iter=2 -r --cpulim=6 --memlim=1024M

Instances , Throughput , Responsetime , MEM_MEAN , CPU_MEAN , CPU_MIN , CPU_MAX , MEM_MIN , MEM_MAX , CLUSTER_MEM% , CLUSTER_CPU% , CPU_REQ , MEM_REQ , CPU_LIM , MEM_LIM , WEB_ERRORS 
1 ,  303.45 , 859.833 , 381.562 , 0.843015 , .7173769627897064 , 1.056741097974608 , 379 , 383 , 17.3531 , 11.9104 ,  ,  , 6 , 1024M , 0
Run , CPU_REQ , MEM_REQ , CPU_LIM , MEM_LIM , Throughput , Responsetime , WEB_ERRORS , CPU , CPU_MIN , CPU_MAX , MEM , MEM_MIN , MEM_MAX
0 ,  ,  , 6 , 1024M , 292.7 , 902 , 0   ,.804033 , .7608657374370931 , .8546878904635039    , 381.475 , 381 , 381 
1 ,  ,  , 6 , 1024M , 301.3 , 852 , 0   ,.855328 , .7566787056455377 , .9630022468676405    , 382.357 , 382 , 382 
2 ,  ,  , 6 , 1024M , 306.3 , 853 , 0   ,.848681 , .7536206260415063 , 1.0542050633967124   , 383.164 , 383 , 383 
0 ,  ,  , 6 , 1024M , 292.7 , 902 , 0   ,.804033 , .7608657374370931 , .8546878904635039    , 381.475 , 381 , 381 
1 ,  ,  , 6 , 1024M , 301.3 , 852 , 0   ,.855328 , .7566787056455377 , .9630022468676405    , 382.357 , 382 , 382 
2 ,  ,  , 6 , 1024M , 306.3 , 853 , 0   ,.848681 , .7536206260415063 , 1.0542050633967124   , 383.164 , 383 , 383 
0 ,  ,  , 6 , 1024M , 306.9 , 850 , 0   ,.852797 , .7862797676675743 , 1.056741097974608    , 380.188 , 379 , 380 
1 ,  ,  , 6 , 1024M , 306.5 , 851 , 0   ,.836347 , .7173769627897064 , 1.0144354095885815   , 380.874 , 380 , 380 
2 ,  ,  , 6 , 1024M , 307.0 , 851 , 0   ,.860906 , .7605042563124672 , .9858358875572207    , 381.312 , 381 , 381
kusumachalasani commented 3 years ago

All the manual runs for bayesian generated config and baseline data together:

Config Throughput Responsetime MEM_MEAN CPU_MEAN CPU_MIN CPU_MAX MEM_MIN MEM_MAX CLUSTER_MEM% CLUSTER_CPU% CPU_REQ MEM_REQ CPU_LIM MEM_LIM WEB_ERRORS
Baseline                              
No req- no lim 325.4125 789.125 510.28625 0.93041525 0.573297563116829 1.2669579143434 487 518 17.6051 12.6668         0
no req-max lim 303.45 859.833 381.562 0.843015 0.717376962789706 1.05674109797461 379 383 17.3531 11.9104     6 1024M 0
Bayesian generated config                              
4.19c-687.5M 302.825 841.125 344.214 0.895 0.736317426432321 1.31416497687714 329 349 17.4404 12.0238 4.19 687.53M 4.19 687.53M 0
4.19c-625.5M 304.9375 857 328.2225 0.84257325 0.607586791782802 1.08837654123294 322 331 17.1166 11.4159 4.19 625.52M 4.19 625.52M 0
4.45c-622.68M 275.817 864.333 343.916 0.885766 0.747251552564334 1.19276653944799 325 363 17.1786 11.9567 4.45 622.68M 4.45 622.68M 0
agarwalrounak commented 3 years ago

No requests, no limits

./run-petclinic-openshift.sh -s towelled.os.fyre.ibm.com -e result -u 400 -d 60 -w 5 -m 3 -i 1 --iter=2 -r 

Instances , Throughput , Responsetime , MEM_MEAN , CPU_MEAN , CPU_MIN , CPU_MAX , MEM_MIN , MEM_MAX , CLUSTER_MEM% , CLUSTER_CPU% , CPU_REQ , MEM_REQ , CPU_LIM , MEM_LIM , WEB_ERRORS 
1 ,  302.5 , 863.667 , 493.112 , 0.885572 , .6543647228723505 , 1.1169717643528645 , 487 , 498 , 17.4214 , 12.1321 ,  ,  ,  ,  , 0
Run , CPU_REQ , MEM_REQ , CPU_LIM , MEM_LIM , Throughput , Responsetime , WEB_ERRORS , CPU , CPU_MIN , CPU_MAX , MEM , MEM_MIN , MEM_MAX
0 ,  ,  ,  ,  , 303.3 , 862 , 0 ,.760344 , .6543647228723505 , .989999913793653 , 487.621 , 487 , 488 
1 ,  ,  ,  ,  , 303.4 , 862 , 0 ,.881221 , .8152317027554571 , 1.0040345081127346   , 488.303 , 488 , 488 
2 ,  ,  ,  ,  , 301.5 , 871 , 0 ,.894192 , .7974622681844135 , .9948416704696477    , 489.028 , 488 , 489 
0 ,  ,  ,  ,  , 303.3 , 862 , 0 ,.760344 , .6543647228723505 , .989999913793653 , 487.621 , 487 , 488 
1 ,  ,  ,  ,  , 303.4 , 862 , 0 ,.881221 , .8152317027554571 , 1.0040345081127346   , 488.303 , 488 , 488 
2 ,  ,  ,  ,  , 301.5 , 871 , 0 ,.894192 , .7974622681844135 , .9948416704696477    , 489.028 , 488 , 489 
0 ,  ,  ,  ,  , 302.8 , 862 , 0 ,.924877 , .7995203366420238 , 1.1076631036424334   , 497.356 , 497 , 497 
1 ,  ,  ,  ,  , 300.9 , 862 , 0 ,.925219 , .8252969050967732 , 1.1169717643528645   , 497.935 , 497 , 498 
2 ,  ,  ,  ,  , 303.1 , 863 , 0 ,.927577 , .840379850466752 , 1.0589093613505123    , 498.43 , 498 , 498
kusumachalasani commented 3 years ago

The latest data of no requests - no limits is within range of all other configurations and is as expected. Please create 2 different files for baseline and manual run data. baseline.csv should consists of

Config Throughput Responsetime MEM_MEAN CPU_MEAN CPU_MIN CPU_MAX MEM_MIN MEM_MAX CLUSTER_MEM% CLUSTER_CPU% CPU_REQ MEM_REQ CPU_LIM MEM_LIM WEB_ERRORS
No req- no lim 302.5 863.667 493.112 0.885572 0.65436472287235 1.11697176435286 487 498 17.4214 12.1321         0
no req-max lim 303.45 859.833 381.562 0.843015 0.717376962789706 1.05674109797461 379 383 17.3531 11.9104     6 1024M 0

manuals.csv should consists of

Config Throughput Responsetime MEM_MEAN CPU_MEAN CPU_MIN CPU_MAX MEM_MIN MEM_MAX CLUSTER_MEM% CLUSTER_CPU% CPU_REQ MEM_REQ CPU_LIM MEM_LIM WEB_ERRORS
4.19c-687.5M 302.825 841.125 344.214 0.895 0.736317426432321 1.31416497687714 329 349 17.4404 12.0238 4.19 687.53M 4.19 687.53M 0
4.19c-625.5M 304.9375 857 328.2225 0.84257325 0.607586791782802 1.08837654123294 322 331 17.1166 11.4159 4.19 625.52M 4.19 625.52M 0
4.45c-622.68M 275.817 864.333 343.916 0.885766 0.747251552564334 1.19276653944799 325 363 17.1786 11.9567 4.45 622.68M 4.45 622.68M 0

As this PR has the data related to TPEmultivariate , create a folder with the name TPE_multivariate inside the algocomparisons folder and place the all 3 files - experiment.csv, baseline.csv and manuals.csv For the other algorithms data, create a folder with that name, and update the data in the respective algorithm folder.