ssh2 does not set mtime/atime (scp_send64, setstat, fsetstat)

[hakaishi]

Documentation says:

scp_send64(self, path, int mode, libssh2_uint64_t size, time_t mtime, time_t atime)

But mtime and atime seem to be ignored. I tried copying from the examples, but it still won't work. My authentication method is by the way userauth_password.

It did work with paramiko, so it shouldn't be an issue with the ssh server.

By the way: I would consider to work around this using setstat(path, attrs), but I'm not sure how I would have to create the SFTPAttributes object. It would be nice to have an example or more documentation on that.

[hakaishi]

Btw.: Using python 3.8.2 on Ubuntu 20.04

[pkittenis]

Hey,

mtime and atime are passed onto libssh2 - whether they do anything depends on libssh2. SFTPAttributes is a python class typical python code to create object and set attributes.

[hakaishi]

I just took a peek into the source code. I didn't find anything suspicious yet. Looking at the manpage, it should be working... No issues reported on the GitHub project either... It would be nice if you could confirm that it is a pure ssh2 library problem. I could open an issue over there then.

[hakaishi]

I've never worked with cdef and the like. The function definitions look like I could simply do things like

fileinfo = stat(path_to_file)
attrs = SFTPAttributes()
attrs.mtime(fileinfo.st_mtime)

and so on... It would be convenient if I could simply initialize the object using the stat result of the file 😇

[hakaishi]

I tried the following (with full paths though):

fifo = stat(".bashrc")
attr = SFTPAttributes()
attr.filesize = fifo.st_size
print(".bashrc", datetime.fromtimestamp(fifo.st_mtime))
attr.atime = fifo.st_atime
attr.mtime = fifo.st_mtime
attr.permissions = fifo.st_mode
attr.gid = fifo.st_gid
attr.uid = fifo.st_uid
self.connection.setstat(".bashrc", attr)
t = self.connection.stat(".bashrc")
print(datetime.fromtimestamp(t.atime),  datetime.fromtimestamp(t.mtime))

first print with local file says: ".bashrc 2019-07-08 08:10:56.616019" second print with remote file says: "2020-08-17 22:38:00 2020-08-17 22:45:40"

EDIT: I also tried it with relative paths (hard coded) with the same negative result

[hakaishi]

I also tried the sftp_handle.fsetstat, but that also doesn't seem to work. Just in case, I also used a vm with mx linux using python 3.7, but it's also not working over there (sent file from vm to local pc).

[pkittenis]

Will try and reproduce in C when have some time, not high priority atm.

[pkittenis]

With 0.23.0 try:

from os import stat
from ssh2.sftp import LIBSSH2_SFTP_ATTR_UIDGID, LIBSSH2_SFTP_ATTR_ACMODTIME, \
    LIBSSH2_SFTP_ATTR_PERMISSIONS

f_stat = stat(".bashrc")
attrs = sftp.stat(_file)
attrs.flags = LIBSSH2_SFTP_ATTR_UIDGID | \
  LIBSSH2_SFTP_ATTR_ACMODTIME | \
  LIBSSH2_SFTP_ATTR_PERMISSIONS
attrs.atime = f_stat.st_atime
attrs.mtime = f_stat.st_mtime
attrs.permissions = f_stat.st_mode
attrs.gid = f_stat.st_gid
attrs.uid = f_stat.st_uid
sftp.setstat(_file, attrs)

[hakaishi]

Looks that I can't import the three attributes. Are you sure about the location "ssh2.sftp"? EDIT: The import shows an error, but it does compile. Testing right now.

[hakaishi]

I tried the following first (using the sftp_handle):

attr = SFTPAttributes(os_stat)
attr.flags = LIBSSH2_SFTP_ATTR_UIDGID | LIBSSH2_SFTP_ATTR_PERMISSIONS | LIBSSH2_SFTP_ATTR_PERMISSIONS
attr.atime = os_stat.st_atime
attr.mtime = os_stat.st_mtime
attr.permissions = os_stat.st_mode
attr.gid = os_stat.st_gid
attr.uid = os_stat.st_uid
file_handle.fsetstat(attr)

This still doesn't work. Then I tried the code above. It does work, but I will have to make one more network transaction. Uploading multiple files will slow down heavily.

EDIT: This works too:

attr = SFTPAttributes()
attr.flags = LIBSSH2_SFTP_ATTR_UIDGID | LIBSSH2_SFTP_ATTR_PERMISSIONS | LIBSSH2_SFTP_ATTR_PERMISSIONS
attr.atime = os_stat.st_atime
attr.mtime = os_stat.st_mtime
attr.permissions = os_stat.st_mode
attr.gid = os_stat.st_gid
attr.uid = os_stat.st_uid
conn.setstat("myfile", attrs)

[pkittenis]

SFTPAttributes(os_stat)

SFTPAttributes does not take arguments.

attr = SFTPAttributes()
attr.atime = os_stat.st_atime
<..>

Setting attributes individually from an OS stat works fine, attr.flags just need to be set correctly depending on what is being changed by sftp.setstat, meaning need LIBSSH2_SFTP_ATTR_ACMODTIME set if changing times and so on.

Setting time with scp_send64 have not tested in C yet, if have some example code that would be really useful. There is an scp example in libssh2/examples in the repository.

[hakaishi]

If you mean Python code, I might manage something tomorrow.

What I meant to say earlier is that conn.setstat works while sftp_handle.fsetstat doesn't.

[pkittenis]

The python code shown for fsetstat does not set the ACMODTIME flag.

Should be:

attr.flags = LIBSSH2_SFTP_ATTR_UIDGID | \
  LIBSSH2_SFTP_ATTR_PERMISSIONS | \
  LIBSSH2_SFTP_ATTR_ACMODTIME
<..>
file_handle.fsetstat(attr)

C code to reproduce scp behaviour would be great, so can debug and raise issue with libssh2, or python code to see if the issue is in ssh2-python or libssh2.

[hakaishi]

Ah! I misplaced a flag! 😅

I'm not too familiar with C, but I'll see what I can do.

[hakaishi]

My python code would look like this (time stamps are ignored though):

finfo = stat(file)
chan = conn.session.scp_send64(
    "%s/%s" % (destination, basename(file)),
    finfo.st_mode & 0o777,
    finfo.st_size,
    finfo.st_mtime, finfo.st_atime)
with open(file, 'rb') as lofi:
    while True:
        data = lofi.read(1024 * 1024)
        if not data:
            break
        else:
            _, sz = chan.write(data)

As for the C code... I didn't got to it, but this might help: https://github.com/libssh2/libssh2/blob/master/example/scp_write.c I found some docu over here

[hakaishi]

Anyways, I can confirm fsetstat and setstat working. The access and modification time is set correctly.

There is a strange thing though... It seems I can import the necessary three flags from ssh2.sftp. Pycharm says that it can't find their references. Looking into the sftp.py, there is no such flags. I suppose they are inside sftp.c and sftp.pyx...

[pkittenis]

The ATTR flags are new in 0.23.0. The IDE probably needs to re-index the new package version. There is no sftp.py in ssh2-python, the imported module is a dynamic library (.so | ldd).

Will try to reproduce in C, not very high priority atm.