Should use MLSD command when the server supports it

[falemagn]

Hi,

gFTP currently uses the LIST command to retrieve directory contents, but appends to it non-standard "parameters", such as -aL: this breaks those servers that abide by the letter of the RFC and stops working on all the other servers the moment a file/directory named "-aL" exists in the current working directory. gFTP should probably at least retry to issue LIST without the appended "-aL" options, when that command fails.

However, most of the FTP servers nowadays support the MLSD command, standardized by RFC 3659, and I believe gFTP should use that for directory listing, only falling back to LIST if MLSD is not supported.

[wdlkmpx]

-a hidden file = .file

Testing several lightweight and old ftp servers, I found one that requires -a to list the .hidden files (ftpdmin.exe), other servers always send .hidden files. Other servers never list .hidden files. I found that Windows servers may send wrong user, group and/or permissions. See https://github.com/masneyb/gftp/commit/0536e898e3a4d84142e23fcad4cb57b012db7f00

gftp itself doesn't need to send -a, gftp_match_filespec() filters out hidden files if Show hidden files is unticked, some servers may not send the full file list without -a, and some servers will never send a complete list ignoring .hidden files and other files, i.e vsftpd

[falemagn]

"-a" is not a standard option, servers are not required to implement support for it and, as said, it fails even in those servers who support it, the moment a file named "-a" exists in the directory being listed.

gFTP completely fails listing directories with the new FileZilla Server, for instance.

RFC 3659 provides a standard way of listing directories which works with all the major current FTP servers.

[wdlkmpx]

I actually agree with everything you say. After some more tests I found that some servers indeed recognize -a, and will not list hidden .files unless that argument is provided.

Public servers send .hidden files, so it might be a matter of configuration, too many options, too many servers, too many apps.

-L: CWD: I found that symlinks work without this parameter in all servers that support symlinks, I click on a symlink to a directory and the target directory is "opened". Only the most barebones servers misbehave (busybox ftpd).

-L: Files: gftp downloads the target files, everything is OK. When the server is faulty (busybox ftpd), gftp tries CWD first, then RETR, and the target file is downloaded.

I think both of these params should be disabled by default so that LIST is standard (in new installations without gftprc). The -L param should be removed indeed..

There is a file rfc959.c,

[wdlkmpx]

Ok the issue is partially fixed, default config values...

# Show hidden files in the listboxes
show_hidden_files=1

# LIST -a: this may break some FTP servers. Some servers list hidden
# files by default
ftp_list_a=0

[wdlkmpx]

MLSD, if I replace LIST by MLSD, I get these responses by public FTP servers (FTP bookmarks), and local servers (apps not quite up to date - Debian Stretch)

MLSD
500 Unknown command.
Invalid response '5' received from server.

MLSD
550 Permission denied.
Invalid response '5' received from server.

Adding support for that will require a bit of research from my part

[wdlkmpx]

I found a page with rants about the FTP protocol, and apparently LIST -al was/is used by konqueror and probably other linux and unix clients by default so it's likely that most servers that don't support extra args just ignore them after acknowledging the fact that there are clients that do crazy things.

So I guess gftp should send LIST -a or LIST -al and what response code should it expect in order to retry without -al? 450 is one reply to trigger the retry.

[wdlkmpx]

There is a way to know if the server supports MLSD, with the FEAT command ( https://datatracker.ietf.org/doc/html/rfc2389 )

I'll try to add code to parse the response and set flags, variables, etc. to modify the client behavior according to that info.

OPTS LIST should provide info about the params accepted by LIST, but I always get error 501

Connected to localhost:21
220 (vsFTPd 3.0.3)
211-Features:
 AUTH SSL
 AUTH TLS
 EPRT
 EPSV
 MDTM
 PASV
 PBSZ
 PROT
 REST STREAM
 SIZE
 TVFS
211 End

Looking up ftp.freebsd.org
FEAT
211-Features:
 EPRT
 EPSV
 MDTM
 PASV
 REST STREAM
 SIZE
 TVFS
 UTF8
211 End

ftp.bit.nl
211-Features:
 UTF8
 EPRT
 SITE SYMLINK
 EPSV
 SITE UTIME
 MDTM
 SITE RMDIR
 TVFS
 MFMT
 LANG en-US.UTF-8*;en-US
 SIZE
 SITE MKDIR
 MFF modify;UNIX.group;UNIX.mode;
 REST STREAM
 MLST modify*;perm*;size*;type*;unique*;UNIX.group*;UNIX.mode*;UNIX.owner*;
211 End

[wdlkmpx]

MLSD (MLST) support has been implemented and is used when available LIST -al is used when MLSD is not supported by the server...

Gftp will fall back to LIST if an error response is received

So far all servers behave as expected, MLSD: ok, LIST -al ok

(LIST: All FTP servers seem to ignore -L (dereference symlink), there must be a reason for that)