MAGE

Memgraph Advanced Graph Extensions :crystal_ball:

This open-source repository contains all available user-defined graph analytics modules and procedures that extend the Cypher query language, written by the team behind Memgraph and its users. You can find and contribute implementations of various algorithms in multiple programming languages, all runnable inside Memgraph. This project aims to give everyone the tools they need to tackle the most challenging graph problems.

Introduction to query modules with MAGE

Memgraph introduces the concept of query modules, user-defined procedures that extend the Cypher query language. These procedures are grouped into modules that can be loaded into Memgraph. How to run them can be seen on their official documentation. When started, Memgraph will automatically attempt to load the query modules from all *.so and *.py files it finds in the default directory defined with flag --query-modules-directory.

Further reading

If you want more info about MAGE, check out the official MAGE Documentation.

Algorithm proposition

Furthermore, if you have an algorithm proposition, please fill in the survey on mage.memgraph.com.

Community

Make sure to check out the Memgraph community and join us on a survey of streaming graph algorithms! Drop us a message on the channels below:

Overview

• Memgraph Advanced Graph Extensions :crystal_ball:
  • Introduction to query modules with MAGE
  • Further reading
  • Algorithm proposition
  • Community
• Overview
• Memgraph compatibility
• How to install MAGE?
  • 1. Use MAGE with Docker
  • a) Get MAGE from Docker Hub
  • b) Install MAGE with a Docker build of the repository
  • 2. Installing MAGE on Linux distro from source
  • Prerequisites
• Running MAGE
• MAGE Spells
• Advanced configuration
• Testing MAGE
• Contributing
• Code of Conduct
• Feedback

Memgraph compatibility

How to install MAGE?

There are two options to install MAGE.

1) For the Docker installation, you only need Docker installed.

2) To build from source, you will need to install a few things first. Jump to section #2 to check for installation details.

After the installation, you will be ready to query Memgraph and use MAGE modules. Make sure to have one of the querying platforms installed as well.

1. Use MAGE with Docker

a) Get MAGE from Docker Hub

1. This command downloads the image from Docker Hub and runs Memgraph preloaded with MAGE modules:

docker run -p 7687:7687 -p 7444:7444 memgraph/memgraph-mage

2 Install MAGE with Docker build of the repository

0. a Make sure that you have cloned the MAGE Github repository and positioned yourself inside the repo in your terminal:

git clone --recurse-submodules https://github.com/memgraph/mage.git && cd mage

0. b Download Memgraph from our official download site inside your cloned MAGE repository. Set ${MEMGRAPH_VERSION} to the latest release of Memgraph, and ${ARCHITECTURE} to your system architecture (amd64 or arm64):

curl -L "https://download.memgraph.com/memgraph/v${MEMGRAPH_VERSION}/debian-11/memgraph_${MEMGRAPH_VERSION}-1_${ARCHITECTURE}.deb" > memgraph-${ARCHITECTURE}.deb

or this one if you are on arm64:

curl -L "https://download.memgraph.com/memgraph/v${MEMGRAPH_VERSION}/debian-11-aarch64/memgraph_${MEMGRAPH_VERSION}-1_arm64.deb" > memgraph-arm64.deb

1. To build the MAGE image run the following command where you set ${architecture} to your system architecture (amd64 or arm64):

DOCKER_BUILDKIT=1 docker buildx build \
--tag memgraph-mage:prod \
--target prod \
--platform linux/${architecture} \
--file Dockerfile.release \
--load .

This will build any new algorithm added to MAGE, and load it inside Memgraph.

2. Start the container with the following command and enjoy Memgraph with MAGE:

docker run --rm -p 7687:7687 -p 7444:7444 --name mage memgraph-mage

NOTE: if you made any changes while the MAGE Docker container was running, you will need to stop it and rebuild the whole image, or you can copy the mage directory inside the Docker container and do the rebuild from there. To learn more about development with MAGE and Docker, visit the documentation.

2. Installing MAGE on Linux distro from source

Note: This step is more suitable for local development.

Prerequisites

• Linux based Memgraph package you can download here. We offer Ubuntu, Debian and CentOS based Memgraph packages. To install, proceed to the following site.
• To build and install MAGE query modules you will need:
  • libcurl4
  • libpython3.9
  • libssl-dev
  • libboost-all-dev
  • openssl
  • build-essential
  • make
  • cmake
  • curl
  • g++
  • python3
  • python3-pip
  • python3-setuptools
  • python3-dev
  • clang
  • unixodbc

Since Memgraph needs to load MAGE's modules, there is the setup script to help you. With it, you can build the modules so that Memgraph can load them on start up.

Before you start, don't forget to clone MAGE with --recurse-submodules flag:

git clone --recurse-submodules https://github.com/memgraph/mage.git && cd mage

Run the following command to install Rust and Python dependencies:

curl https://sh.rustup.rs -sSf | sh -s -- -y \
&& export PATH="/root/.cargo/bin:${PATH}" \
&& python3 -m  pip install -r /mage/python/requirements.txt \
&& python3 -m  pip install -r /mage/python/tests/requirements.txt \
&& python3 -m  pip install torch-sparse torch-cluster torch-spline-conv torch-geometric torch-scatter -f https://data.pyg.org/whl/torch-1.12.0+cu102.html \

Now you can run the following command to compile and copy the query modules to the /usr/lib/memgraph/query_modules path:

python3 setup build -p /usr/lib/memgraph/query_modules

It will generate a mage/dist directory and copy the modules to the /usr/lib/memgraph/query_modules directory.

Note that query modules are loaded into Memgraph on startup so if your instance was already running you will need to execute the following query inside one of the querying platforms to load them: CALL mg.load_all();

Running MAGE

This is an example of running the PageRank algorithm on a simple graph. You can find more details on the documentation page.

// Create the graph from the image below

CALL pagerank.get()
YIELD node, rank;

Graph input	MAGE output

MAGE Spells

Advanced configuration

• Advanced query module directories setup
• Developing MAGE with Docker

Testing MAGE

To test that everything is built, loaded, and working correctly, a python script can be run. Make sure that the Memgraph instance with MAGE is up and running.

# Running unit tests for C++ and Python
python3 test_unit

# Running end-to-end tests
python3 test_e2e

Furthermore, to test only specific end-to-end tests, you can add argument -k with substring referring to the algorithm that needs to be tested. To test a module named <query_module>, you would have to run python3 test_e2e -k <query_module> where <query_module> is the name of the specific module you want to test.

# Running specific end-to-end tests
python3 test_e2e -k weakly_connected_components

Contributing

We encourage everyone to contribute with their own algorithm implementations and ideas. If you want to contribute or report a bug, please take a look at the contributions guide.

Code of Conduct

Everyone participating in this project is governed by the Code of Conduct. By participating, you are expected to uphold this code. Please report unacceptable behavior to tech@memgraph.com.

Feedback

Your feedback is always welcome and valuable to us. Please don't hesitate to post on our Community Forum.