maxisam / mgob

MongoDB dockerized backup agent. Runs schedule backups with retention, S3 & SFTP upload, notifications, instrumentation with Prometheus and more.
https://maxisam.github.io/mgob/
MIT License
152 stars 22 forks source link

Unable to send backups to SSH server #129

Open remialvado opened 2 months ago

remialvado commented 2 months ago

Hi,

I try to setup a new instance of mgob and send backups to an SFTP server I've just setup. My mgob configuration is the following one :

scheduler:
  # run every day at 6:00
  cron: "0 6 * * *"
  # number of backups to keep locally
  retention: 14
  # backup operation timeout in minutes
  timeout: 60
target:
  host: "mongo"
  port: 27017
  username: "<myuser>"
  password: "<mypassword>"
sftp:
  host: sftp.tooling.mysite.dev
  port: 2222
  username: staging1
  private_key: /etc/ssh/ssh_key
  dir: backup

and my docker-compose is the following one :

  mgob:
    image: maxisam/mgob:2.0.24
    container_name: "staging1_mgob"
    volumes:
      - ./config/mgob/plan:/config
      - /data/mgob/storage:/storage
      - /data/mgob/tmp:/tmp
      - /data/mgob/data:/data
      # add ssh key to access sftp using "staging1" user
      - ./config/mgob/ssh/id_rsa:/etc/ssh/ssh_key:ro

The SFTP server has been created in another container using https://hub.docker.com/r/atmoz/sftp with the following configuration :

  sftp:
    image: atmoz/sftp:alpine
    container_name: "tooling_sftp"
    volumes:
      # add an SSH key for host
      - ./config/sftp/_host/ssh_host_rsa_key:/etc/ssh/ssh_host_rsa_key
      - ./config/sftp/_host/ssh_host_ed25519_key:/etc/ssh/ssh_host_ed25519_key
      # mount data directory for "production" user and add ssh key
      - /data/sftp/production/backup:/home/production/backup
      - ./config/sftp/production/id_rsa.pub:/home/production/.ssh/keys/id_rsa.pub:ro
      # mount data directory for "staging1" user and add ssh key
      - /data/sftp/staging1/backup:/home/staging1/backup
      - ./config/sftp/staging1/id_rsa.pub:/home/staging1/.ssh/keys/id_rsa.pub:ro
    ports:
      - "2222:22"
    command: production::1001 staging1::1002

My SSH private key on mgob container seems to have the correct permissions even if the user 1004 seems wierd to me :

a596db31311c:/# ls -la /etc/ssh/
total 584
drwxr-xr-x    1 root     root          4096 Sep 23 14:39 .
drwxr-xr-x    1 root     root          4096 Sep 23 14:39 ..
-rw-r--r--    1 root     root        573991 Jul  2 15:54 moduli
-rw-r--r--    1 root     root          1531 Jul  2 15:54 ssh_config
-rw-------    1 1004     1004          3422 Sep 23 13:56 ssh_key

On my laptop, when I try to connect to the FTP server, everything works well but on mbox, when I run a dump, I got the following error :

a596db31311c:/# curl -X POST http://localhost:8090/backup/staging1
{"error":"SSH dial to sftp.tooling.mysite.dev:2222 failed: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none], no supported methods remain"}

Any idea?

remialvado commented 2 months ago

I've also tried to change the owner of the ssh_key file to root:root on host, restart the mgob container and start another backup but the error is still the same

remialvado commented 1 month ago

I've found my mistake : private_key must be named privateKey. It is written in README.md but not in https://github.com/maxisam/mgob/blob/main/.document/BACKUP_PLAN.md I'll make a PR to fix it once everything is working on my side.