Receiving 'invalid PASV response format' error

[mmolinac]

After you fixed issue https://github.com/AlexAkulov/clickhouse-backup/issues/283 through PR https://github.com/AlexAkulov/clickhouse-backup/pull/288 , we continue with our tests uploading a backup through FTP protocol. You'll find below the config file we're using. Credentials have been redacted for security reasons:

---
general:
    remote_storage: ftp
    disable_progress_bar: false
    backups_to_keep_local: 0
    backups_to_keep_remote: 0
clickhouse:
    username: "backup_user"
    password: "12341234"
    host: "localhost"
    port: 9000
    data_path: ""
    skip_tables:
        - system.*
    timeout: 5m
    freeze_by_part: false
    secure: false
    skip_verify: false
ftp:
    address: "u111111.your-storagebox.de:21"
    timeout: 2m
    username: "u111111"
    password: "222222"
    tls: false
    path: "clickhouse01"  # This path has been created beforehand
    compression_format: tar
    compression_level: 1
    debug: true

Then we issue the following command (see error at the end):

# LOG_LEVEL=debug clickhouse-backup --config /etc/clickhouse-backup/config.ftp.yml upload test-backup-full-20211025-1852
2021/10/25 19:04:48 debug SELECT value FROM `system`.`build_options` where name='VERSION_INTEGER'
2021/10/25 19:04:48 debug SELECT * FROM system.disks;
2021/10/25 19:04:48 debug SELECT * FROM system.disks;
2021/10/25 19:04:48 debug SELECT value FROM `system`.`build_options` where name='VERSION_INTEGER'
2021/10/25 19:04:48 debug SELECT * FROM system.disks;
2021/10/25 19:04:48 debug FTP::getConnectionFromPool(Walk) active=0 idle=0
220 ProFTPD Server (Hetzner Backup) [a:b:c:d::2]
USER u111111
331 Password required for u111111
PASS 222222
230 User u111111 logged in
FEAT
211-Features:
 AUTH TLS
 CCC
 CLNT
 EPRT
 EPSV
 HOST
 LANG ko-KR.UTF-8;ko-KR;it-IT.UTF-8;it-IT;es-ES.UTF-8;es-ES;bg-BG.UTF-8;bg-BG;zh-CN.UTF-8;zh-CN;ja-JP.UTF-8;ja-JP;ru-RU.UTF-8;ru-RU;zh-TW.UTF-8;zh-TW;fr-FR.UTF-8;fr-FR;en-US.UTF-8;en-US
 MDTM
 MFF modify;UNIX.group;UNIX.mode;
 MFMT
 MLST modify*;perm*;size*;type*;unique*;UNIX.group*;UNIX.groupname*;UNIX.mode*;UNIX.owner*;UNIX.ownername*;
 PBSZ
 PROT
 RANG STREAM
 REST STREAM
 SIZE
 SSCN
 TVFS
 UTF8
211 End
TYPE I
200 Type set to I
OPTS UTF8 ON
200 UTF8 set to on
PASV
227 Entering Passive Mode (2,233,252).
2021/10/25 19:04:48 debug FTP::returnConnectionToPool(Walk) active=1 idle=0
QUIT
2021/10/25 19:04:48 error invalid PASV response format

We can log in to the FTP server with different FTP clients without any issue. Even lftp from the same server we're trying to upload the backup from.

[Slach]

Is hetzner backup runned on ipv6? Try to use ipv4

[mmolinac]

Turned out that they have both A and AAAA DNS records. I forced the usage of the IPv4 address and it worked! Any hint of the reason for that behaviour?

[Slach]

@mmolinac look like issue related to https://github.com/jlaffaye/ftp incorrect behavior

227 Entering Passive Mode (2,233,252). should be 227 Entering Passive Mode (a:b:c:d::2,233,252).

[mmolinac]

@Slach looks like @jlaffaye has a hint of how to solve it, here

[Slach]

@mmolinac thanks a lot for reporting and figure out, check https://github.com/AlexAkulov/clickhouse-backup/tree/v1.2.1 and you are welcome to reopen the issue, if you require

[Slach]

@jawr JFYI