bit-team / backintime

Back In Time - An easy-to-use backup tool for GNU Linux using rsync in the back
https://backintime.readthedocs.io
GNU General Public License v2.0
2.03k stars 201 forks source link

Different ports for rsync server and ssh terminal (Synology) #1430

Open FrancisH92 opened 1 year ago

FrancisH92 commented 1 year ago

Hi, On my Synology DS740 I use a different port for the ssh rsync server and the ssh terminal. It seems BiT is using the same port to check login without password mode and to access the rsync server, and hence I have the following errors depending on the port setting in BiT:

backintime.txt

emtiu commented 1 year ago

Thanks for your report! I'm fairly certain that this isn't implemented yet. It sounds like a resonable feature request, but the dev team is small, and this will probably not be a high priority to be honest.

buhtz commented 1 year ago

I'm asking just for my own learning curve in understanding how rsync (with BIT) works.

Does BIT use an rsync server somehow?

I thought when using a SSH profile snapshot there is no rsync-server but an ssh-server (at remote machine) involved. The rsync-client does access the remote machine via SSH and somehow fetch the data from there. In my understanding on remote there never is an rsync instance involved. Am I wrong?

Germar commented 1 year ago

Does BIT use an rsync server somehow?

No, it doesn't

I thought when using a SSH profile snapshot there is no rsync-server but an ssh-server (at remote machine) involved. The rsync-client does access the remote machine via SSH and somehow fetch the data from there.

That's correct. Rsync starts a remote instance through ssh and communicate with it over the ssh-tunnel

aryoda commented 1 year ago

@FrancisH92 With "rsync server" do you refer to this Synology dialog (where have you changed the port number?)?

Synology rsync service config

The ssh access itself is configured here (where you use port 22):

Synology SSH service config

I did never try this combination (since BiT does not support the rsync server), but I am wondering if it would work at all with different ports (would require rsync server to start another sshd instance I guess).

@FrancisH92 Does the rsync server with this configuration work with other tools (or via cmd line)?

aryoda commented 1 year ago

Synology says:

https://kb.synology.com/en-global/DSM/help/DSM/AdminCenter/file_rsync?version=7

  • If the rsync service is encrypted (that is, having a specified SSH encryption port), the system will directly apply DSM users' usernames and passwords as their rsync accounts.
  • If the rsync service is unencrypted, you need to create corresponding rsync accounts for them.

So I'd say Enable rsync account is not required to be checked in the DSM config GUI...

If have just played around with the rsync service on the Synology NAS a little bit:

If I start the rsync service on my Synology (e.g. using the ssh encryption port setting 1234 in above dialog) this process is started (ps ax - grep -i rsync):

/usr/bin/rsync --daemon

The daemon is listening at the default port 873 so the configured port "1234" is NOT the rsync port:

netstat -tunlp | grep rsync
tcp        0      0 0.0.0.0:873             0.0.0.0:*               LISTEN      28832/rsync         
tcp6       0      0 :::873                  :::*                    LISTEN      28832/rsync  

If I check which ports are used now via netstat -tunlp I can see:

...
tcp        0      0 0.0.0.0:1234            0.0.0.0:*               LISTEN      -  
tcp6       0      0 :::1234                 :::*                    LISTEN      -                   
tcp6       0      0 :::22                   :::*                    LISTEN 
...

so someone is listening at the configured port...

Nevertheless I cannot find the configured port 1234 in the rsyncd.conf:

$ cat /etc/rsyncd.conf 
refuse options = acls
#motd file = /etc/rsyncd.motd
#log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
use chroot = no

But a recursive grep shows the port in this file (which I do not know):

$ cat synoinfo.conf | grep rsync
rsync_sshd_port="1234"
rsync_account="no"

So currently I do not yet understand how Synology is using ssh if another port in the rsync service config GUI is specified :-(

Edit: It looks like Synology is spawning a second sshd instance to listen to the rsync ssh port 1234 but I still don't know where the port number is configured or passed via cmd line so that the second sshd instance uses this port... (the proc sys shows only sshd: /usr/bin/sshd -D [listener] 0 of 10-100 startups as cmdline and I cannot find any documentation about these arguments)

aryoda commented 1 year ago

@FrancisH92 Even though the rsync server may not be supported directly by BiT it may possibly still be used implicitly when a certain syntax for the backup target is specified (possibly :: after the host name).

Could you please

  1. Append a screen shot of your BiT setting that you are using for the backup (you many anonymize parts if required)
  2. Append the output of backintime-qt --debug after you have tried to start the backup (please try this separately for both variations - port 22 and port 22632).
FrancisH92 commented 1 year ago

Hi Aryoda,

yes, I checked the "Enable rsync service" and entered a specific port (12345). I as well checked the "Enable SSH service" and kept kept the standard port 22. I use this configuration to log into the Synology using SSH port 22 and I daily run a home made backup using rsync port 12345. Both are working fine.

Best regards

Le 03/05/2023 à 12:20, aryoda a écrit :

@FrancisH92 https://github.com/FrancisH92 With "rsync server" do you refer to this Synology dialog (where you have changed the port number)?

Synology rsync service config https://user-images.githubusercontent.com/11374410/235889689-734090a3-e877-4553-b2ce-91ec44bbab6b.png

The ssh access itself is configured here (where you use port 22):

Synology SSH service config https://user-images.githubusercontent.com/11374410/235890020-8365955d-f33e-432a-8213-0be2e4f4ef24.png

I did never try this combination (since BiT does not support the rsync server), but I am wondering if it would work at all with different ports (would require rsync server to start another sshd instance I guess).

@FrancisH92 https://github.com/FrancisH92 Does the rsync server with this configuration work with other tools (or via cmd line)?

— Reply to this email directly, view it on GitHub https://github.com/bit-team/backintime/issues/1430#issuecomment-1532780880, or unsubscribe https://github.com/notifications/unsubscribe-auth/A7QZGGXYGSVUMCDN2VARH6LXEIWQRANCNFSM6AAAAAAXRCOPEU. You are receiving this because you were mentioned.Message ID: @.***>

-- Francis Hauber 44 rue du Docteur le Savoureux 92290 Châtenay-Malabry +33 6 22 56 70 00 @.***

FrancisH92 commented 1 year ago

Hi,

I tested again using port 22 and below I have appended the BiT screen and the debug output. My understanding is

DEBUG: [common/sshtools.py:256 SSH.startSshAgent] ssh-agent already running. Skip starting a new one. DEBUG: [common/sshtools.py:411 SSH.unlockSshAgent] Private key /home/francis/.ssh/id_rsa is already unlocked in ssh agent DEBUG: [common/sshtools.py:681 SSH.checkPingHost] Check ping host DEBUG: [common/sshtools.py:700 SSH.checkPingHost] Host syno.fhauber is available DEBUG: [common/mount.py:582 SSH.checkFuse] Check fuse DEBUG: [common/sshtools.py:256 SSH.startSshAgent] ssh-agent already running. Skip starting a new one. DEBUG: [common/sshtools.py:321 SSH.unlockSshAgent] Remove private key /home/francis/.ssh/id_rsa from ssh agent DEBUG: [common/sshtools.py:340 SSH.unlockSshAgent] Add private key /home/francis/.ssh/id_rsa to ssh agent DEBUG: [common/sshtools.py:350 SSH.unlockSshAgent] Password available: True DEBUG: [common/sshtools.py:567 SSH.checkKnownHosts] Check known hosts file DEBUG: [common/sshtools.py:580 SSH.checkKnownHosts] Host syno.fhauber was found in known hosts file DEBUG: [common/sshtools.py:423 SSH.checkLogin] Check login DEBUG: [common/sshtools.py:599 SSH.checkRemoteFolder] Check remote folder DEBUG: [common/sshtools.py:626 SSH.checkRemoteFolder] Call command: ssh -o ServerAliveInterval=240 -o LogLevel=Error -o Ident ityFile=/home/francis/.ssh/id_rsa -p 22 @. d=0;test -e "NetBackup" || d=1;test $d -eq 1 && mkdir "NetBackup "; err=$?;test $d -eq 1 && exit $err;test -d "NetBackup" || exit 11;test -w "NetBackup" || exit 12;test -x "NetBackup" || exi t 13;exit 20 INFO: [common/sshtools.py:661 SSH.checkRemoteFolder] Create remote folder NetBackup DEBUG: [common/sshtools.py:735 SSH.checkRemoteCommands] Check remote commands DEBUG: [common/sshtools.py:788 SSH.checkRemoteCommands] Check rsync command: ['rsync', '--recursive', '--times', '--devices', '--specials', '--hard-links', '--human-readable', '-s', '--links', '--perms', '--executability', '--group', '--owner', '--rs h=ssh -o ServerAliveInterval=240 -o LogLevel=Error -o IdentityFile=/home/francis/.ssh/id_rsa -p 22', '/tmp/tmpyd2jhpfd/a', 'f @.:NetBackup/tmp_DEMUTS/'] DEBUG: [common/sshtools.py:797 SSH.checkRemoteCommands] rsync command returned error: Permission denied, please try again. rsync: connection unexpectedly closed (0 bytes received so far) [sender] rsync error: error in rsync protocol data stream (code 12) at io.c(231) [sender=3.2.7]


Le 03/05/2023 à 14:52, aryoda a écrit :

@FrancisH92 https://github.com/FrancisH92 Even though the rsync server may not be supported directly by BiT it may possibly still be used implicitly when a certain syntax for the backup target is specified (possibly |::| after the host name).

Could you please

  1. Append a screen shot of your BiT setting that you are using for the backup (you many anonymize parts if required)
  2. Append the output of |backintime-qt --debug| after you have tried to start the backup (please try this separately for both variations - port 22 and port 22632).

— Reply to this email directly, view it on GitHub https://github.com/bit-team/backintime/issues/1430#issuecomment-1532977507, or unsubscribe https://github.com/notifications/unsubscribe-auth/A7QZGGUY2OPL4TIHKGWJJKLXEJIIFANCNFSM6AAAAAAXRCOPEU. You are receiving this because you were mentioned.Message ID: @.***>

-- Francis Hauber 44 rue du Docteur le Savoureux 92290 Châtenay-Malabry +33 6 22 56 70 00 @.***

aryoda commented 1 year ago

@FrancisH92

Could you please also provide the debug output when using the rsync daemon ssh port so that I can see the differences? THX :-)

I daily run a home made backup using rsync port 12345. Both are working fine.

Would it be possible to show me the (anonymized) rsync command line you are using here so that I can compare it to the BiT-generated command line to find the differences. BiT allows to add rsync arguments for a profile "by hand" in the GUI (settings > expert options > paste additional options to rsync)...

'@.***:NetBackup/tmp_DEMUTS/'

so the configured target uses one colon only which means rsync does not use the rsync daemon of Synology (see https://www.atlantic.net/vps-hosting/how-to-use-rsync-copy-sync-files-servers/ for the syntax of the rsync protocoll which requires two colons, e.g.: rsync [-options] SOURCE user@x.x.x.x::DESTINATION).

So perhaps specifying the rsync daemon ssh port 12345 and using two colons in the BiT profile could solve the "permission" denied problem... (why using port 22 fails is not yet obvious to me...).

FrancisH92 commented 1 year ago

Hi,

please find the debug output below.

The test you suggested, connecting on the rsybc-daemon port (12345),  failed, in my view because the Synology's rsync-daemon doesn't understand the ssh and ssh-copy-id commands, so login obviously failed, as well as ssh-copy-id.

DEBUG: [common/mount.py:73 Mount.init] pw-cache is not running DEBUG: [common/mount.py:81 Mount.init] Call command: /usr/bin/backintime pw-cache start DEBUG: [common/sshtools.py:256 SSH.startSshAgent] ssh-agent already running. Skip starting a new one. DEBUG: [common/sshtools.py:411 SSH.unlockSshAgent] Private key /home/francis/.ssh/id_rsa is already unlocked in ssh agent DEBUG: [common/sshtools.py:681 SSH.checkPingHost] Check ping host DEBUG: [common/sshtools.py:700 SSH.checkPingHost] Host syno.fhauber is available DEBUG: [common/mount.py:582 SSH.checkFuse] Check fuse DEBUG: [common/sshtools.py:256 SSH.startSshAgent] ssh-agent already running. Skip starting a new one. DEBUG: [common/sshtools.py:321 SSH.unlockSshAgent] Remove private key /home/francis/.ssh/id_rsa from ssh agent DEBUG: [common/sshtools.py:340 SSH.unlockSshAgent] Add private key /home/francis/.ssh/id_rsa to ssh agent DEBUG: [common/sshtools.py:350 SSH.unlockSshAgent] Password available: True DEBUG: [common/sshtools.py:567 SSH.checkKnownHosts] Check known hosts file DEBUG: [common/sshtools.py:580 SSH.checkKnownHosts] Host syno.fhauber was found in known hosts file DEBUG: [common/sshtools.py:423 SSH.checkLogin] Check login ERROR: [qt/settingsdialog.py:1597 SettingsDialog.saveProfile] L'authentification sans mot de passe pour @.*** à échoué. Consultez 'man backintime' pour de plus amples instructions.

Permission denied, please try again.

DEBUG: [common/sshtools.py:1059 sshCopyId] Call command "ssh-copy-id -i /home/francis/.ssh/id_rsa.pub -p 12345 @. hauber" ERROR: [common/sshtools.py:1072 sshCopyId] Failed to copy ssh-key "/home/francis/.ssh/id_rsa.pub" to @.": [1] /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/francis/.ssh/id_rsa.pub" /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys Permission denied, please try again.

Le 03/05/2023 à 23:58, aryoda a écrit :

@FrancisH92 https://github.com/FrancisH92

Could you please also provide the debug output when using the rsync daemon ssh port so that I can see the differences? THX :-)

I daily run a home made backup using rsync port 12345. Both are
working fine.

Would it be possible to show me the (anonymized) rsync command line you are using here so that I can compare it to the BiT-generated command line to find the difference. BiT allows to add rsync arguments for a profile "by hand" in the GUI...

'/*@*/.***:NetBackup/tmp_DEMUTS/'

so the configured target uses one colon only which means rsync does not use the rsync daemon of Synology (see https://www.atlantic.net/vps-hosting/how-to-use-rsync-copy-sync-files-servers/ for the syntax of the rsync protocoll which requires two colons, e.g.: |rsync [-options] SOURCE @.***::DESTINATION|).

So perhaps specifying the rsync daemon ssh port 12345 and using two colons in the BiT profile could solve the "permission" denied problem... (why using port 22 fails is not yet obvious to me...).

— Reply to this email directly, view it on GitHub https://github.com/bit-team/backintime/issues/1430#issuecomment-1533806384, or unsubscribe https://github.com/notifications/unsubscribe-auth/A7QZGGSVDKKEY2ETEU5DEQLXELIJPANCNFSM6AAAAAAXRCOPEU. You are receiving this because you were mentioned.Message ID: @.***>

-- Francis Hauber 44 rue du Docteur le Savoureux 92290 Châtenay-Malabry +33 6 22 56 70 00 @.***

aryoda commented 1 year ago

@FrancisH92 I have put this issue on my todo list and will try it on my own Synology NAS to find out if and how it could work.

Since we have a bunch of other open issues with higher priority this may take a while (our "developer bandwidth" is limited ;-)