Memgraph replication IN_MEMORY_ANALYTICAL and IN_MEMORY_TRANSACTIONAL switch bug.

[antejavor]

The things we often recommend for faster import are IN_MEMORY ANALYTICAL, moving to ANALYTICAL for import, and then getting back to TRANSACTIONAL once the import is complete.

Turns out that it affects the replication setup. Below is the script that simulate following:

1. Setup the replication
2. Move to ANALYTICAL
3. Import the data
4. Move back to TRANSACTIONAL
5. There is no data in ASYNC and SYNC replicas. (bug)

Full context and discussion: https://discord.com/channels/842007348272169002/1235670651403239425/1235918233429606451

#!/bin/bash

# Start the MAIN instance
docker run -d --name memgraph-main -p 7687:7687 memgraph/memgraph:2.16.0

# Start REPLICA instance 1 (SYNC mode)
docker run -d --name memgraph-replica1 -p 7688:7687 memgraph/memgraph:2.16.0

# Start REPLICA instance 2 (ASYNC mode)
docker run -d --name memgraph-replica2 -p 7689:7687 memgraph/memgraph:2.16.0

# Wait for instances to be ready
echo "Waiting for instances to start up..."
sleep 5

# Get IP addresses of the containers
MAIN_IP=$(docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' memgraph-main)
REPLICA1_IP=$(docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' memgraph-replica1)
REPLICA2_IP=$(docker inspect -f '{{range.NetworkSettings.Networks}}{{.IPAddress}}{{end}}' memgraph-replica2)

echo "MAIN instance IP: $MAIN_IP"
echo "REPLICA 1 IP: $REPLICA1_IP"
echo "REPLICA 2 IP: $REPLICA2_IP"

# Demote instances to REPLICA role and set replication port
echo "Setting replication roles..."
echo "SET REPLICATION ROLE TO REPLICA WITH PORT 10000;" | docker exec -i memgraph-replica1 mgconsole
echo "SET REPLICATION ROLE TO REPLICA WITH PORT 10000;" | docker exec -i memgraph-replica2 mgconsole

# Register replicas on the MAIN instance
echo "Registering replicas..."
echo "REGISTER REPLICA REP1 SYNC TO \"$REPLICA1_IP\";" | docker exec -i memgraph-main mgconsole
echo "REGISTER REPLICA REP2 ASYNC TO \"$REPLICA2_IP\";" | docker exec -i memgraph-main mgconsole

# Change MAIN instance to IN_MEMORY_ANALYTICAL mode
echo "Changing MAIN instance to IN_MEMORY_ANALYTICAL mode..."
echo "STORAGE MODE IN_MEMORY_ANALYTICAL;" | docker exec -i memgraph-main mgconsole

# Create dummy nodes
echo "Creating dummy nodes..."
echo "CREATE (n:Dummy {id: 1});" | docker exec -i memgraph-main mgconsole
echo "CREATE (n:Dummy {id: 2});" | docker exec -i memgraph-main mgconsole
echo "CREATE (n:Dummy {id: 3});" | docker exec -i memgraph-main mgconsole

# Revert MAIN instance to IN_MEMORY_TRANSACTIONAL mode
echo "Reverting MAIN instance to IN_MEMORY_TRANSACTIONAL mode..."
echo "STORAGE MODE IN_MEMORY_TRANSACTIONAL;" | docker exec -i memgraph-main mgconsole

# Check the status of the registered replicas
echo "Checking registered replicas..."
echo "SHOW REPLICAS;" | docker exec -i memgraph-main mgconsole

echo "Replication setup complete."

# Wait for key press
echo "Press any key to remove instances..."
read -n 1 -s

# Remove instances
echo "Removing instances..."
docker stop memgraph-main
docker rm memgraph-main
docker stop memgraph-replica1
docker rm memgraph-replica1
docker stop memgraph-replica2
docker rm memgraph-replica2

echo "Instances removed."

[katarinasupe]

Currently, the best flow is:

1. Move to ANALYTICAL
2. Import the data
3. Move back to TRANSACTIONAL
4. Setup the replication

(Replication is not supported in in-memory analytical mode and probably that messes up with the flow stated in the issue)