search

digint / btrbk

Tool for creating snapshots and remote backups of btrfs subvolumes
https://digint.ch/btrbk/
GNU General Public License v3.0
1.71k stars 123 forks source link

Penalty from the SSH server if multiple shell commands are sent in sequence: `penalty: connections without attempting authentication` #605

Closed goekce closed 4 hours ago

goekce commented 4 hours ago

I have the following scenario: A remote backup server that runs btrfs (X) and collects backup from the computer Y using ssh.

A disk failure happens in X, so snapshots are made in Y but cannot be collected by X. After the failure recovery, X wants to delete about 50 snapshots in Y and issues many ssh commands. A couple succeed but most of them not:

X logs:

....many delete lines
Nov 15 11:00:22 X btrbk[62251]: [delete] target: example.de:/btr_pool/rootfs.20241110T1700                                                                                                                                                   
Nov 15 11:00:22 X btrbk[62251]: ERROR: Failed to delete subvolume: example.de:/btr_pool/rootfs.20241110T1700                                                                                                                                 
Nov 15 11:00:22 X btrbk[62251]: ERROR: ... SSH command failed (exitcode=255)                                                                                                                                                               
Nov 15 11:00:22 X btrbk[62251]: ERROR: ... sh: ssh -i '/root/id_ed25519' root@example.de 'btrfs subvolume delete '\''/btr_pool/rootfs.20241110T1700'\'''                                                                     
Nov 15 11:00:22 X btrbk[62251]: ERROR: ... kex_exchange_identification: read: Connection reset by peer                                                                                                                                     
Nov 15 11:00:22 X btrbk[62251]: ERROR: ... Connection reset by 1.1.1.1 port 22
.... many error lines

Y logs:

Nov 15 10:59:04 aydos sshd[340]: drop connection #0 from [X_IP]:19017 on [Y_IP]:22 penalty: connections without attempting authentication                                                                                  
...about 40 of them

My workaround was to delete the snapshots manually because I could not find an option to throttle ssh connections in btrbk. I also tried increasing MaxStartups and MaxSessions in SSH server config. Can someone confirm this behavior?

goekce commented 4 hours ago

Because MaxStartups and MaxSessions in SSH server did not help, it may be related to the corporate firewall which I cannot experiment with. So I lean towards closing this issue to leave space for others.