search

anyproto / any-sync-dockercompose

docker-compose for testing any-sync
MIT License
234 stars 29 forks source link

Files not synching #79

Open Sacrifice4135 opened 3 days ago

Sacrifice4135 commented 3 days ago

Have you read a contributing guide?

Current Behavior

The Anytype Applications are synching and backing up to the Anytype Sync Node but only text und structure, no images, files, videos. The Files arent listened under "Remote Storage".

Expected Behavior

The Anytype Apps should sync and backup every data.

Steps To Reproduce

  1. Create Ubuntu Server 22.04 LTS Virtual Maschine on ESXi
  2. Add a second disk with 100GB
  3. Mount the new disk unter /data/anytype and add it to the /etc/fstab file to mount it automatically on every boot.
  4. Install Docker and add it to autostart
  5. Clone Git Repo: git clone https://github.com/anyproto/any-sync-dockercompose.git
  6. adjust env.common File: AWS_ACCESS_KEY_ID=minio_access_key AWS_SECRET_ACCESS_KEY=minio_secret_key MINIO_BUCKET=minio-bucket MINIO_PORT=9000 EXTERNAL_MINIO_PORT=${MINIO_PORT} MINIO_WEB_PORT=9001 EXTERNAL_MINIO_WEB_PORT=${MINIO_WEB_PORT}

ANY_SYNC_NODE_VERSION=prod ANY_SYNC_FILENODE_VERSION=prod ANY_SYNC_COORDINATOR_VERSION=prod ANY_SYNC_CONSENSUSNODE_VERSION=prod ANY_SYNC_TOOLS_VERSION=latest

MONGO_VERSION=7.0.2 REDIS_VERSION=7.2.0-v6 MINIO_VERSION=RELEASE.2024-01-16T16-07-38Z

EXTERNAL_LISTEN_HOST="127.0.0.1"

Change the ip address of the host

EXTERNAL_LISTEN_HOSTS="192.168.22.176"

ANY_SYNC_NODE_1_HOST=any-sync-node-1 ANY_SYNC_NODE_1_PORT=1001 ANY_SYNC_NODE_1_ADDRESSES=${ANY_SYNC_NODE_1_HOST}:${ANY_SYNC_NODE_1_PORT} ANY_SYNC_NODE_1_QUIC_PORT=1011 ANY_SYNC_NODE_1_QUIC_ADDRESSES=${ANY_SYNC_NODE_1_HOST}:${ANY_SYNC_NODE_1_QUIC_PORT}

ANY_SYNC_NODE_2_HOST=any-sync-node-2 ANY_SYNC_NODE_2_PORT=1002 ANY_SYNC_NODE_2_ADDRESSES=${ANY_SYNC_NODE_2_HOST}:${ANY_SYNC_NODE_2_PORT} ANY_SYNC_NODE_2_QUIC_PORT=1012 ANY_SYNC_NODE_2_QUIC_ADDRESSES=${ANY_SYNC_NODE_2_HOST}:${ANY_SYNC_NODE_2_QUIC_PORT}

ANY_SYNC_NODE_3_HOST=any-sync-node-3 ANY_SYNC_NODE_3_PORT=1003 ANY_SYNC_NODE_3_ADDRESSES=${ANY_SYNC_NODE_3_HOST}:${ANY_SYNC_NODE_3_PORT} ANY_SYNC_NODE_3_QUIC_PORT=1013 ANY_SYNC_NODE_3_QUIC_ADDRESSES=${ANY_SYNC_NODE_3_HOST}:${ANY_SYNC_NODE_3_QUIC_PORT}

ANY_SYNC_COORDINATOR_HOST=any-sync-coordinator ANY_SYNC_COORDINATOR_PORT=1004 ANY_SYNC_COORDINATOR_ADDRESSES=${ANY_SYNC_COORDINATOR_HOST}:${ANY_SYNC_COORDINATOR_PORT} ANY_SYNC_COORDINATOR_QUIC_PORT=1014 ANY_SYNC_COORDINATOR_QUIC_ADDRESSES=${ANY_SYNC_COORDINATOR_HOST}:${ANY_SYNC_COORDINATOR_QUIC_PORT}

Change Space Limits

ANY_SYNC_COORDINATOR_DEFAULT_LIMITS_SPACE_MEMBERS_READ=1000000 ANY_SYNC_COORDINATOR_DEFAULT_LIMITS_SPACE_MEMBERS_WRITE=1000000 ANY_SYNC_COORDINATOR_DEFAULT_LIMITS_SHARED_SPACES_LIMIT=1000000

ANY_SYNC_FILENODE_HOST=any-sync-filenode ANY_SYNC_FILENODE_PORT=1005 ANY_SYNC_FILENODE_ADDRESSES=${ANY_SYNC_FILENODE_HOST}:${ANY_SYNC_FILENODE_PORT} ANY_SYNC_FILENODE_QUIC_PORT=1015 ANY_SYNC_FILENODE_QUIC_ADDRESSES=${ANY_SYNC_FILENODE_HOST}:${ANY_SYNC_FILENODE_QUIC_PORT}

Change size of the disk were anytype sync node is installed

ANY_SYNC_FILENODE_DEFAULT_LIMIT=107374182400

ANY_SYNC_CONSENSUSNODE_HOST=any-sync-consensusnode ANY_SYNC_CONSENSUSNODE_PORT=1006 ANY_SYNC_CONSENSUSNODE_ADDRESSES=${ANY_SYNC_CONSENSUSNODE_HOST}:${ANY_SYNC_CONSENSUSNODE_PORT} ANY_SYNC_CONSENSUSNODE_QUIC_PORT=1016 ANY_SYNC_CONSENSUSNODE_QUIC_ADDRESSES=${ANY_SYNC_CONSENSUSNODE_HOST}:${ANY_SYNC_CONSENSUSNODE_QUIC_PORT}

MONGO_1_PORT=27001 MONGO_CONNECT=mongodb://mongo-1:${MONGO_1_PORT} MONGO_REPLICA_SET=rs0 MONGO_URL=${MONGO_CONNECT}/?replicaSet=${MONGO_REPLICA_SET}

REDIS_HOST=redis REDIS_PORT=6379 REDIS_URL="redis://${REDIS_HOST}:${REDIS_PORT}?dial_timeout=3&read_timeout=6s"

Optional change the storage directory

STORAGE_DIR="./storage"

  1. Allow Ports in UFW:

    TCP-Ports

    sudo ufw allow 443/tcp sudo ufw allow 8000/tcp sudo ufw allow 8080/tcp sudo ufw allow 1001/tcp sudo ufw allow 1002/tcp sudo ufw allow 1003/tcp sudo ufw allow 1004/tcp sudo ufw allow 1005/tcp sudo ufw allow 1006/tcp sudo ufw allow 27001/tcp sudo ufw allow 6379/tcp sudo ufw allow 9000/tcp sudo ufw allow 9001/tcp

UDP-Ports

sudo ufw allow 1011/udp sudo ufw allow 1012/udp sudo ufw allow 1013/udp sudo ufw allow 1014/udp sudo ufw allow 1015/udp sudo ufw allow 1016/udp

Reload Firewall

sudo ufw reload

  1. Anytype start: make start

Environment

- OS: Ubuntu Server 22.04 LTS
- Version: Ubuntu 22.04.4 LTS (GNU/Linux 5.15.0-113-generic x86_64)

Anything else?

I have tried to fix this issue. When the server is booted up and I run "make down" and after the containers are shutted down "make start", the sync works. But it cant to normal that I need to restart Anytype manually every time.

Sacrifice4135 commented 3 days ago

Github formatted the .env.common File weird. Sorry for that

Sacrifice4135 commented 3 days ago

image image

See, storage is empty, even with uploaded pictures in the notes.

Sacrifice4135 commented 2 days ago

I have tested with different Linux OS and have found one thing in common:

The filesync in the latest version 5.0 from last week doesnt work whatever im doing. It simply doesnt work.

The Filesync with the Pre-Release 4.1.0 works most of the time. After a Server Reboot, I still need to restart the Anytype Sync node, but the files are synchronising.

corius498 commented 1 day ago

I have the same issue on Debian, no picture/file sync with the newest version. New setup on Server with the prod images doesn't work.