search

ldbc / ldbc_snb_bi

Reference implementations for the LDBC Social Network Benchmark's Business Intelligence (BI) workload
https://ldbcouncil.org/benchmarks/snb-bi
Apache License 2.0
35 stars 21 forks source link

Q6 parameter tuning produces bimodal runtime #28

Closed jackwaudby closed 2 years ago

jackwaudby commented 2 years ago

SF1 on Neo4j

szarnyasg commented 2 years ago

Still occurs with SF100 on Umbra: image

Will investigate.

szarnyasg commented 2 years ago

image

szarnyasg commented 2 years ago

New Umbra results: image

It changes its speed midway through the query results. Mysterious:

100|6|<All_by_Myself>|2.960888385772705
100|6|<Sikh_Empire>|2.9917385578155518
100|6|<Lee_Hyung-taik>|2.8916690349578857
100|6|<German_colonial_empire>|3.0150580406188965
100|6|<French_Union>|2.966428756713867
100|6|<Hohenzollern-Sigmaringen>|3.016329288482666
100|6|<Principality_of_Antioch>|3.056137800216675
100|6|<Lady_Marmalade>|2.9981882572174072
100|6|<Sweetheart_of_the_Rodeo>|2.977882146835327
100|6|<Seventeen_Provinces>|2.9676432609558105
100|6|<The_Division_Bell>|2.9501442909240723
100|6|<Band_of_Gypsys>|2.9744067192077637
100|6|<Exile_on_Main_St.>|3.0082907676696777
100|6|<The_Eminem_Show>|2.9782350063323975
100|6|<Khanate_of_Kazan>|2.995375871658325
100|6|<Paul_Keres>|2.804177761077881
100|6|<George_Bentham>|2.986233711242676
100|6|<Ayutthaya_Kingdom>|2.999314785003662
100|6|<Colonel_Bogey_March>|2.97910213470459
100|6|<Bad_Romance>|2.9682321548461914
100|6|<Manny_Ramirez>|2.8330485820770264
100|6|<Appetite_for_Destruction>|2.950690507888794
100|6|<Roll_Over_Beethoven>|2.987703561782837
100|6|<French_Somaliland>|2.970956802368164
100|6|<Frisia>|3.0215377807617188
100|6|<Beautiful_Day>|2.987278461456299
100|6|<Xiongnu>|3.0159225463867188
100|6|<Czechoslovak_Socialist_Republic>|3.0125980377197266
100|6|<In_Rainbows>|3.0285255908966064
100|6|<Coolio>|2.9007976055145264
100|6|<Prince-Bishopric_of_Liège>|2.9751648902893066
100|6|<Lordship_of_Ireland>|2.9993438720703125
100|6|<Malacca_Sultanate>|2.9772186279296875
100|6|<International_Energy_Agency>|3.0030710697174072
100|6|<France_in_the_Middle_Ages>|2.9829931259155273
100|6|<OK_Computer>|2.9722697734832764
100|6|<Hotel_California>|3.0158257484436035
100|6|<Dilip_Kumar>|2.922252893447876
100|6|<Ingrid_van_Houten-Groeneveld>|2.9765617847442627
100|6|<Saxe-Altenburg>|2.9577927589416504
100|6|<Both_Sides,_Now>|2.9714181423187256
100|6|<Sark>|2.984053134918213
100|6|<Nevermind>|3.0136663913726807
100|6|<Magical_Mystery_Tour>|3.0152852535247803
100|6|<In_da_Club>|2.972830057144165
100|6|<Eleanor_Rigby>|3.032717704772949
100|6|<Tristan_da_Cunha>|1.100492000579834
100|6|<Nevis>|1.145141363143921
100|6|<Hephthalite>|1.100501537322998
100|6|<Girls_Just_Want_to_Have_Fun>|1.1108145713806152
100|6|<HIStory:_Past,_Present_and_Future,_Book_I>|1.1020581722259521
100|6|<Beggars_Banquet>|1.0573923587799072
100|6|<Whole_Lotta_Love>|1.016268014907837
100|6|<We_Are_the_Champions>|1.1898419857025146
100|6|<Electric_Ladyland>|1.1952219009399414
100|6|<TV_Asahi>|1.0509657859802246
100|6|<Kurt_Gödel>|0.9553818702697754
100|6|<Duchy_of_Styria>|1.0333359241485596
100|6|<Long_Tall_Sally>|1.0468580722808838
100|6|<Saint_Barthélemy>|1.1301372051239014
100|6|<West_Pakistan>|1.1161653995513916
100|6|<Afsharid_dynasty>|1.1011931896209717
100|6|<Viceroyalty_of_the_Río_de_la_Plata>|1.0664868354797363
100|6|<Western_European_Union>|1.1036224365234375
100|6|<Daisy_Bell>|1.0456349849700928
100|6|<Please_Please_Me>|1.1395072937011719
100|6|<This_Land_Is_Your_Land>|1.1952917575836182
100|6|<Proud_Mary>|1.1803250312805176
100|6|<Duchy_of_Saxony>|1.1208789348602295
100|6|<Duchy_of_Oldenburg>|1.1204955577850342
100|6|<Forever_Changes>|1.076242208480835
100|6|<Netherlands_New_Guinea>|1.0603418350219727
100|6|<All_Along_the_Watchtower>|1.110386610031128
100|6|<Duchy_of_Carinthia>|1.1797137260437012
100|6|<Caucasian_Iberia>|1.0239927768707275
100|6|<Twist_and_Shout>|3.4671342372894287
100|6|<Buyid_dynasty>|1.0964553356170654
100|6|<Adjara>|1.046600103378296
100|6|<Blue_Suede_Shoes>|1.100799322128296
100|6|<Happy_Days_Are_Here_Again>|1.1228039264678955
100|6|<Land_of_Confusion>|3.4804182052612305
100|6|<John_Edward_Gray>|1.1098382472991943
100|6|<With_The_Beatles>|1.0991203784942627
100|6|<Layla>|1.1653060913085938
100|6|<Lili_Marleen>|1.1223292350769043
100|6|<German_New_Guinea>|1.1075897216796875
100|6|<Axis:_Bold_as_Love>|1.2089695930480957
100|6|<Empire_of_Nicaea>|1.0991196632385254
100|6|<Luis_Horna>|0.6490669250488281
100|6|<Ai_Sugiyama>|1.1478979587554932
100|6|<Good_Vibrations>|1.105499029159546
100|6|<I_Am..._Sasha_Fierce>|1.0490732192993164
100|6|<Republic_of_Upper_Volta>|1.1633460521697998
100|6|<I_Will_Survive>|1.2057280540466309
100|6|<Aquarius/Let_the_Sunshine_In>|1.1842882633209229
100|6|<Basutoland>|1.1128120422363281
100|6|<Republic_of_Serbian_Krajina>|1.1324725151062012
100|6|<Free_Bird>|1.0663585662841797
100|6|<Ghaznavids>|1.1613640785217285
100|6|<Pala_Empire>|1.1460661888122559
szarnyasg commented 2 years ago

SF10, Umbra: image

szarnyasg commented 2 years ago

SF100: there are still a few outliers: 4 results are ~3 seconds while the rest are between 0.4-1.1s. I deem this "good enough".

Umbra|100|6|<All_by_Myself>|0.9247198104858398
Umbra|100|6|<Sikh_Empire>|3.4095795154571533
Umbra|100|6|<Lee_Hyung-taik>|0.5875051021575928
Umbra|100|6|<German_colonial_empire>|1.0431232452392578
Umbra|100|6|<French_Union>|0.9401223659515381
Umbra|100|6|<Hohenzollern-Sigmaringen>|1.0453007221221924
Umbra|100|6|<Principality_of_Antioch>|1.0567591190338135
Umbra|100|6|<Lady_Marmalade>|1.01389479637146
Umbra|100|6|<Sweetheart_of_the_Rodeo>|0.9861981868743896
Umbra|100|6|<Seventeen_Provinces>|0.9527859687805176
Umbra|100|6|<The_Division_Bell>|0.8860018253326416
Umbra|100|6|<Band_of_Gypsys>|0.9331519603729248
Umbra|100|6|<Exile_on_Main_St.>|0.9906127452850342
Umbra|100|6|<The_Eminem_Show>|0.9545230865478516
Umbra|100|6|<Khanate_of_Kazan>|1.0065109729766846
Umbra|100|6|<Paul_Keres>|0.4055173397064209
Umbra|100|6|<George_Bentham>|0.9571571350097656
Umbra|100|6|<Ayutthaya_Kingdom>|0.9919579029083252
Umbra|100|6|<Colonel_Bogey_March>|0.9542832374572754
Umbra|100|6|<Bad_Romance>|0.9629495143890381
Umbra|100|6|<Manny_Ramirez>|0.4168851375579834
Umbra|100|6|<Appetite_for_Destruction>|0.9124553203582764
Umbra|100|6|<Roll_Over_Beethoven>|0.9899404048919678
Umbra|100|6|<French_Somaliland>|0.9388363361358643
Umbra|100|6|<Frisia>|1.0578241348266602
Umbra|100|6|<Beautiful_Day>|0.9971010684967041
Umbra|100|6|<Xiongnu>|1.0474977493286133
Umbra|100|6|<Czechoslovak_Socialist_Republic>|1.010408639907837
Umbra|100|6|<In_Rainbows>|1.0587704181671143
Umbra|100|6|<Coolio>|3.3972880840301514
Umbra|100|6|<Prince-Bishopric_of_Liège>|0.9660203456878662
Umbra|100|6|<Lordship_of_Ireland>|1.0043213367462158
Umbra|100|6|<Malacca_Sultanate>|0.9547207355499268
Umbra|100|6|<International_Energy_Agency>|1.016528844833374
Umbra|100|6|<France_in_the_Middle_Ages>|0.9816234111785889
Umbra|100|6|<OK_Computer>|0.9375648498535156
Umbra|100|6|<Hotel_California>|1.0105383396148682
Umbra|100|6|<Dilip_Kumar>|0.6070327758789062
Umbra|100|6|<Ingrid_van_Houten-Groeneveld>|0.9871425628662109
Umbra|100|6|<Saxe-Altenburg>|0.9380512237548828
Umbra|100|6|<Both_Sides,_Now>|0.9506633281707764
Umbra|100|6|<Sark>|1.0128898620605469
Umbra|100|6|<Nevermind>|1.0545880794525146
Umbra|100|6|<Magical_Mystery_Tour>|1.0523664951324463
Umbra|100|6|<In_da_Club>|0.9326338768005371
Umbra|100|6|<Eleanor_Rigby>|1.0655412673950195
Umbra|100|6|<Tristan_da_Cunha>|0.980140209197998
Umbra|100|6|<Nevis>|1.0407929420471191
Umbra|100|6|<Hephthalite>|0.9902088642120361
Umbra|100|6|<Girls_Just_Want_to_Have_Fun>|1.006483554840088
Umbra|100|6|<HIStory:_Past,_Present_and_Future,_Book_I>|0.9869022369384766
Umbra|100|6|<Beggars_Banquet>|0.9562199115753174
Umbra|100|6|<Whole_Lotta_Love>|0.9324705600738525
Umbra|100|6|<We_Are_the_Champions>|1.059105396270752
Umbra|100|6|<Electric_Ladyland>|1.068582534790039
Umbra|100|6|<TV_Asahi>|0.953223466873169
Umbra|100|6|<Kurt_Gödel>|0.8820474147796631
Umbra|100|6|<Duchy_of_Styria>|0.951117753982544
Umbra|100|6|<Long_Tall_Sally>|0.9521389007568359
Umbra|100|6|<Saint_Barthélemy>|1.0208027362823486
Umbra|100|6|<West_Pakistan>|1.0053222179412842
Umbra|100|6|<Afsharid_dynasty>|0.9920856952667236
Umbra|100|6|<Viceroyalty_of_the_Río_de_la_Plata>|0.9790570735931396
Umbra|100|6|<Western_European_Union>|1.0007128715515137
Umbra|100|6|<Daisy_Bell>|0.9684865474700928
Umbra|100|6|<Please_Please_Me>|1.0251247882843018
Umbra|100|6|<This_Land_Is_Your_Land>|1.067124843597412
Umbra|100|6|<Proud_Mary>|1.0519218444824219
Umbra|100|6|<Duchy_of_Saxony>|1.0130431652069092
Umbra|100|6|<Duchy_of_Oldenburg>|1.0090436935424805
Umbra|100|6|<Forever_Changes>|0.9728095531463623
Umbra|100|6|<Netherlands_New_Guinea>|0.964174747467041
Umbra|100|6|<All_Along_the_Watchtower>|0.9990003108978271
Umbra|100|6|<Duchy_of_Carinthia>|1.0477867126464844
Umbra|100|6|<Caucasian_Iberia>|0.9264113903045654
Umbra|100|6|<Twist_and_Shout>|3.4154446125030518
Umbra|100|6|<Buyid_dynasty>|0.9788594245910645
Umbra|100|6|<Adjara>|0.9475193023681641
Umbra|100|6|<Blue_Suede_Shoes>|0.9824483394622803
Umbra|100|6|<Happy_Days_Are_Here_Again>|1.0101780891418457
Umbra|100|6|<Land_of_Confusion>|3.4491000175476074
Umbra|100|6|<John_Edward_Gray>|0.9955580234527588
Umbra|100|6|<With_The_Beatles>|0.9867618083953857
Umbra|100|6|<Layla>|1.027984380722046
Umbra|100|6|<Lili_Marleen>|1.00199294090271
Umbra|100|6|<German_New_Guinea>|0.9840898513793945
Umbra|100|6|<Axis:_Bold_as_Love>|1.0724151134490967
Umbra|100|6|<Empire_of_Nicaea>|0.9898257255554199
Umbra|100|6|<Luis_Horna>|0.5912072658538818
Umbra|100|6|<Ai_Sugiyama>|1.0420615673065186
Umbra|100|6|<Good_Vibrations>|1.0015394687652588
Umbra|100|6|<I_Am..._Sasha_Fierce>|0.948204755783081
Umbra|100|6|<Republic_of_Upper_Volta>|1.0453205108642578
Umbra|100|6|<I_Will_Survive>|1.0821566581726074
Umbra|100|6|<Aquarius/Let_the_Sunshine_In>|1.0504038333892822
Umbra|100|6|<Basutoland>|0.9957435131072998
Umbra|100|6|<Republic_of_Serbian_Krajina>|1.0092945098876953
Umbra|100|6|<Free_Bird>|0.9533307552337646
Umbra|100|6|<Ghaznavids>|1.0321276187896729
Umbra|100|6|<Pala_Empire>|1.0154032707214355