antejavor
commented
1 year ago Hi @jhb, thanks for reporting the issue. I am aware that prerequisites are not precise and detailed at the moment. After I post more details on how to run the benchmarks on this issue, I will also update the methodology part. Aldo, these things could be done differently, inside docker, by downloading memgraph etc. As you suggested, I will post steps for the setup we used to get mentioned results.
The steps are a bit tiresome, mainly because we used mgBench as an in-house benchmark tool, and it is tightly coupled with Memgraph. We plan to make these steps a bit easier in the future.
Memgraph part:
- You will need a Linux OS machine (CentOS, Debian or Ubuntu) to compile a Memgraph and Client
- Compile memgraph (for help, look at this short guide [here] (https://memgraph.notion.site/Quick-Start-82a99a85e62a4e3d89f6a9fb6d35626d) take a look at step 3 and 4 before diving in.
- Make sure that in the build configuration step, replace
cmake ..withcmake -DCMAKE_BUILD_TYPE=Release .., since you want to test performance on a Release version. - Compile the whole project because we are using a custom C++ client found here. You can also compile just memgraph and client, but this is easier. If you want, you can use
make -j$(nproc) memgraphfor memgraph andmake -j$(nproc) memgraph__mgbench__clientfor the client. - At this point, locate your binary in the build folder with _Release suffix.
Neo4j part:
- Download the binary version of the 5.1 community edition on their website for the appropriate OS.
- Make sure you have at least JDK 17 on your machine.
- Running the benchmark process will disable auth on Neo4j in the config file.
After that, it comes to running the benchmark. Position yourself in the mgbench directory and call
graph_bench.py
--vendor memgraph /home/memgraph/build/binary
--dataset-group basic
--dataset-size small
--realistic 100 30 70 0 0
--realistic 100 50 50 0 0
--realistic 100 70 30 0 0
--realistic 100 30 40 10 20
--mixed 100 30 0 0 0 70 Of course, replace the path with the Memgraph path to the standalone binary. When you decide to run Neo4j, replace a vendor name with Neo4j, and pass a path to the root neo4j project, not the path to the binary. Like this:
graph_bench.py
--vendor neo4j /home/neo4j-5.2
--dataset-group basic
--dataset-size small
--realistic 100 30 70 0 0
--realistic 100 50 50 0 0
--realistic 100 70 30 0 0
--realistic 100 30 40 10 20
--mixed 100 30 0 0 0 70 If you have any additional questions regarding the benchmark or need help, feel free to ask and let me know by email or another issue :D
jhb
Describe the bug I want to reproduce the benchmarks on https://memgraph.com/benchgraph, but I am not sure how either memgraph nor neo4j are correctly to be installed. The only information available is at https://github.com/memgraph/memgraph/tree/master/tests/mgbench#prerequisites, which only tells me to install them in their binary form.
Expected behavior
I would love to know if setting up the databases in their binary form means that I have to compile memgraph myself, e.g. with what options, or if I should use the docker image, and run the tests inside the docker image? Or in another way? Which one?
It would be great if the setup was described for memgraph (and ideally also for neo4j), in a way that allows to be certain that the setup is correct, and that deviations from the published results on https://memgraph.com/benchgraph are not due to the setup.