search

mongodb / mongo-perf

performance tools for mongodb
351 stars 140 forks source link

PERF-1825 Add microbenchmarks for new pipeline-style findAndModify #64

Closed cswanson310 closed 5 years ago

cswanson310 commented 5 years ago

Three things to note right off the bat:

  1. The tags will have to include some special 'pipeline-update' tag so that we can include them in their own task, which is the plan outlined in this doc
  2. Unlike that document, we didn't port over FindAndModifySortedRemoveIndexed. This was doing a remove-style findAndModify, so there's no pipeline-equivalent to write.
  3. I forgot to give the last workload a new name so there's a duplicate name. Sorry! I'll propose a name inline in a comment or in the next patch.

patch build

For some reason the new workloads aren't showing up in a way that's easy to find? They're in the logs though! Extracted here for your viewing pleasure (heavily redacted repeated runs so that the summary is more clear):

STANDALONE
[2019/05/22 15:26:11.314] Commands.FindAndModifySortedUpdateWithPipeline
 [2019/05/22 15:26:11.314] 1    186.29641919092987
 [2019/05/22 15:26:11.314] 2    362.71091819849045
 [2019/05/22 15:26:11.314] 4    697.0769917441629
 [2019/05/22 15:26:11.314] 8    717.899023389813
[2019/05/22 15:26:11.314] Commands.FindAndModifySortedUpdateIndexedWithPipeline
 [2019/05/22 15:26:11.314] 1    5425.8950646973635
 [2019/05/22 15:26:11.314] 2    8922.039695907097
 [2019/05/22 15:26:11.314] 4    15973.616684795097
 [2019/05/22 15:26:11.314] 8    16988.158023405587
[2019/05/22 15:36:16.391] Commands.FindAndModifyGradeAdjustment
 [2019/05/22 15:36:16.391] 1    4686.733793520529
 [2019/05/22 15:36:16.391] 2    9290.685974814052
 [2019/05/22 15:36:16.391] 4    18327.183075043817
 [2019/05/22 15:36:16.391] 8    19835.53710414117
[2019/05/22 15:36:16.391] Commands.FindAndModifyGradeAdjustment (TAKE 2, different test)
 [2019/05/22 15:36:16.391] 1    200.99771589871952
 [2019/05/22 15:36:16.391] 2    396.4742567506537
 [2019/05/22 15:36:16.391] 4    780.7022242618107
 [2019/05/22 15:36:16.391] 8    765.1149129921384

1-NODE REPL SET
[2019/05/22 15:25:11.102] Commands.FindAndModifySortedUpdateWithPipeline
 [2019/05/22 15:25:11.102] 1    184.2334897719441
 [2019/05/22 15:25:11.102] 2    363.258972476639
 [2019/05/22 15:25:11.102] 4    707.8759844431116
 [2019/05/22 15:25:11.102] 8    709.6270407015809
[2019/05/22 15:25:11.102] Commands.FindAndModifySortedUpdateIndexedWithPipeline
 [2019/05/22 15:25:11.102] 1    2405.364755326929
 [2019/05/22 15:25:11.102] 2    3374.784226583285
 [2019/05/22 15:25:11.102] 4    5677.569252548357
 [2019/05/22 15:25:11.102] 8    5806.490948035034
[2019/05/22 15:35:31.829] Commands.FindAndModifyGradeAdjustment
 [2019/05/22 15:35:31.829] 1    3912.093291885916
 [2019/05/22 15:35:31.829] 2    7783.645804418726
 [2019/05/22 15:35:31.829] 4    14879.110849085848
 [2019/05/22 15:35:31.829] 8    15631.19920650157
[2019/05/22 15:35:31.829] Commands.FindAndModifyGradeAdjustment (TAKE 2 - different test)
 [2019/05/22 15:35:31.829] 1    194.524913383434
 [2019/05/22 15:35:31.829] 2    386.1544057979125
 [2019/05/22 15:35:31.829] 4    748.6226622029092
 [2019/05/22 15:35:31.829] 8    769.4243727694427
ksajme commented 5 years ago

All of this seems reasonable to me and I think we have enough coverage considering that we're testing updates (and of course pipelines) more thoroughly elsewhere. I am assuming you ran the last two findAndModifys in the shell to make sure they do what you described.

dalyd commented 5 years ago

A couple quick responses before looking at the code:

  1. I'm disturbed that the results aren't displaying. I can see they were uploaded correctly. They show up in the perf.json section, so it shouldn't be anything you did. Could you ask in #evergreen-users about that, and/or open an EVG ticket?
  2. With the tags, we can add in exclude tags if necessary to separate tests into tasks.
cswanson310 commented 5 years ago

Ok I rebased on top of Jacob's changes from #65. The latest 3 commits show the changes I made after applying the rebase (which had no conflicts since it was different files).

dalyd commented 5 years ago

Ok I rebased on top of Jacob's changes from #65. The latest 3 commits show the changes I made after applying the rebase (which had no conflicts since it was different files).

FYI for future use, with github PR, it's better to merge in changes, than rebase the PR line. It handles the diffs better and will still do the squash and merge correctly at the end.

I'll take a look through in a little bit. From my previous comment it should be quick,