Missing SFTP file attribute constants

[Togtja]

Bug reports

I tried to set permission on a file on a SFTP server. I use normally use FileZilla to interact with the server, but I needed a fast way to change a lot of files, file permission. So I thought I would write a python script.

I noticed 2 things, if I only have read access to the file, I could not change the permission, giving me a ssh2.exceptions.SFTPProtocolError, I can using the same login, change the permission in FileZilla.

The second and more important issue, was that I could not change the permission at all using the code described below: Steps to reproduce:

def chmod(sftp, path, permissions):
    stat = sftp.stat(path)
    print("original permission", oct(stat.permissions))

    stat.permissions = permissions | 0x8000 #I tried without the '|' as well, and I tried with only LIBSSH2_SFTP_S_IRWXU
    print("seting permissions to ", oct(stat.permissions))

    try:
        if sftp.setstat(path, stat) == 0:
            stat = sftp.stat(path)
            print("managed to set permissions to ", oct(stat.permissions))
        else:
            print("failed to set permissions")

    except Exception:
        raise

The out put is this when trying to change the permission of a file from '666' to '755'

original permission 0o100666
seting permissions to  0o100755
managed to set permissions to  0o100666
success

Expected behaviour: I expect that I can change permission on files with only read-access, as long as the authenticated user, has privileges todo so I expected that the new file permission would be '755' after calling setstat with changed permissions Actual behaviour: When I only read access has been set on a file it throws a ssh2.exceptions.SFTPProtocolError When trying to setstat it does not change, even though the setstat() returned a 0 Additional info: I am uncertain of which libssh2 I am running, as I just pip installed ssh2-python and it was up and running, but if I were to hazard a guess, I'd say 1.9.0

[pkittenis]

Thanks for the interest.

stat.permissions should be a mask of LIBSSH2_SFTP_S_* flags. Example from tests:

attrs = sftp.stat(remote_filename)
attrs.permissions = LIBSSH2_SFTP_S_IRUSR
assert sftp.setstat(remote_filename, attrs) == 0
attrs = sftp.stat(remote_filename)
self.assertEqual(attrs.permissions, 33024)

33024 == 100400 in octal (read only).

SFTPProtocolError means invalid attributes were given to setstat. If the flags were not used, this is probably why.

Will need to see code that reproduces the read only file case to help further.

[Togtja]

To be clearer: I only get the SFTPProtocolError error with the read only example:

def chmod(sftp, path, permissions):
    stat = sftp.stat(path)
    print("current permission is", oct(stat.permissions))
    stat.permissions = LIBSSH2_SFTP_S_IRWXU
    print("setting permission to be", oct(stat.permissions))

    try:
        if sftp.setstat(path, stat) == 0:
            stat = sftp.stat(path)
            print("success, new permission is", oct(stat.permissions))
        else:
            print("failed to set stat")
    except Exception:
        raise

"""Set up sftp server stuff"""

chmod(sftp, "test.txt", None)

This throws this error:

current permission is 0o100444
setting permission to be 0o700
Traceback (most recent call last):
  File "c:\Users\labuser\Documents\test_folder\python_ssl.py", line 41, in <module>
    chmod(sftp, "test.txt", None)
  File "c:\Users\labuser\Documents\test_folder\python_ssl.py", line 15, in chmod
    if sftp.setstat(path, stat) == 0:
  File "ssh2\sftp.pyx", line 385, in ssh2.sftp.SFTP.setstat
  File "ssh2\utils.pyx", line 184, in ssh2.utils.handle_error_codes
ssh2.exceptions.SFTPProtocolError

If I make the file with write access (using FileZilla in my case, but I could go ssh and do chmod 666 test.txt as well): running the same code again, I get this output:

current permission is 0o100666
setting permission to be 0o700
success, new permission is 0o100666

Here I use LIBSSH2_SFTP_S_IRWXU as permission, which I did say I tried in the OG post

[pkittenis]

Can you show output of:

In shell:

ls -lh test.txt

In python:

<login code here>
chmod(sftp, "test.txt", None)

Please show complete python code, including login code.

[Togtja]

The setup for the sftp variable looks like this:

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((host, 22))

session = Session()
session.handshake(sock)
session.userauth_password(user, pwd)

sftp = session.sftp_init()

The output from the ls command (login in with same user, pwd as the code)

sftp> ls -lh test.txt
-rw-rw-rw-    ? 0        0              0B Oct 13 14:41 test.txt

[pkittenis]

What is the owner of the file and what is the login user?

? is not a valid user name. The python code is not complete and does not show login user.

The login user needs to have owner or group status in order to change permissions. Exception is raised otherwise.

[Togtja]

The owner of the file should be togtja, but the ls -lh command shows -rw-rw-rw- ? 0 0 0B Oct 13 14:41 test.txt The code is complete, I just hide the personal information such as the host, user and password, but for the sake of argument I can make example variable:

Entire code:

def chmod(sftp, path, permissions):
    stat = sftp.stat(path)
    print("current permission is", oct(stat.permissions))
    stat.permissions = LIBSSH2_SFTP_S_IRWXU
    print("setting permission to be", oct(stat.permissions))

    try:
        if sftp.setstat(path, stat) == 0:
            stat = sftp.stat(path)
            print("success, new permission is", oct(stat.permissions))
        else:
            print("failed to set stat")
    except Exception:
        raise

host = "example.com"
user = "togtja"
pwd = "123password"
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((host, 22))

session = Session()
session.handshake(sock)
session.userauth_password(user, pwd)

sftp = session.sftp_init()

chmod(sftp, "test.txt", None)

output:

current permission is 0o100666
setting permission to be 0o700
success, new permission is 0o100666

I can create a file using ssh2-python so I will try to do that, and comment the output

[pkittenis]

The owner of the file should be togtja, but the ls -lh command shows -rw-rw-rw- ? 0 0 0B Oct 13 14:41 test.txt

According to the shell, it is not. Does chmod 644 test.txt work when logged in as togtja? If not, it will not work with ssh2-python either.

There is a test that creates a file locally and changes permissions with ssh2-python.

Will need to show some code to reproduce an issue to be able to help further.

[Togtja]

Hmm it seems that the sftp server only accepts permissions of "444" and "666", so it's not an issue with (ssh2-python/linssh2). might be because the SFTP server is running on Windows, so that might be the reason for my issues. (But I have no choice) However, I am still not able to change the permission from a read-only, to write only: The file test.txt has 666 permission:

chmod(sftp, "test.txt", 0o444)
chmod(sftp,  "test.txt", 0o666)

The first chmod outputs:

current permission is 0o100666
setting permission to be 0o100444
success, new permission is 0o100444

However it fails on the second one

current permission is 0o100444
setting permission to be 0o100666
error 
Traceback (most recent call last):
  File "ssh2_test.py", line 53, in <module>
    chmod(sftp, "test.txt", 0o666)
  File "ssh2_test.py", line 13, in chmod
    if sftp.setstat(path, stat) == 0:
  File "ssh2\sftp.pyx", line 385, in ssh2.sftp.SFTP.setstat
  File "ssh2\utils.pyx", line 184, in ssh2.utils.handle_error_codes
ssh2.exceptions.SFTPProtocolError

If I do the same in the console:

sftp> ls -l
-rw-rw-rw-   1 user     group           0 Oct 13 12:41 test.txt
sftp> chmod 444 test.txt 
Changing mode on test.txt
sftp> ls -l
-r--r--r--   1 user     group           0 Oct 13 12:41 test.txt
sftp> chmod 666 test.txt 
Changing mode on test.txt
sftp> ls -l
-rw-rw-rw-   1 user     group           0 Oct 13 12:41 test.txt

Here I expect it to behave the same. And I am able to do it using CURL (with libssh2 1.9.0)

[pkittenis]

To be able to help with this please show:

Complete code that reproduces the issue.

Will also need to show login user, user owning file and python code using appropriate masks from LIBSSH2_SFTP_S_*. Ignoring these masks and using hardcoded octal numbers will not work. Particularly across operating systems..

Please also show result of ls -lh test.txt - can use sftp.listdir if not have shell access - after changing permissions with sftp.setstat.

[Togtja]

Alot of this is similar from before, such as login user, the same login user should be the owner of the file, but all files on the file server, including those not made by my login user, are all owned by ?, I think it might have to do with it being a Windows server, but not certain. Also minor improvment so I can transalte octal to LIBSSH2_SFTP_S_*

def chmod(sftp, path, permissions):
    perm = 0
    if permissions & 0o400:
        perm |= LIBSSH2_SFTP_S_IRUSR
    if permissions & 0o200:
        perm |= LIBSSH2_SFTP_S_IWUSR
    if permissions & 0o100:
        perm |= LIBSSH2_SFTP_S_IXUSR
    if permissions & 0o040:
        perm |= LIBSSH2_SFTP_S_IRGRP
    if permissions & 0o020:
        perm |= LIBSSH2_SFTP_S_IWGRP
    if permissions & 0o010:
        perm |= LIBSSH2_SFTP_S_IXGRP
    if permissions & 0o004:
        perm |= LIBSSH2_SFTP_S_IROTH
    if permissions & 0o002:
        perm |= LIBSSH2_SFTP_S_IWOTH
    if permissions & 0o001:
        perm |= LIBSSH2_SFTP_S_IXOTH

    stat = sftp.stat(path)
    print("current permission is", oct(stat.permissions))
    stat.permissions = perm
    print("setting permission to be", oct(stat.permissions))

    try:
        if sftp.setstat(path, stat) == 0:
            stat = sftp.stat(path)
            print("success, new permission is", oct(stat.permissions))
        else:
            print("failed to set stat")
    except SFTPProtocolError as e:
        print("error", e)
        raise

host = "example.com"
user = "togtja"
pwd = "123password"
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((host, 22))

session = Session()
session.handshake(sock)
session.userauth_password(user, pwd)

sftp = session.sftp_init()

chmod(sftp, "test.txt", 0o444)
chmod(sftp, "test.txt", 0o666)

Please also show result of ls -lh test.txt - can use sftp.listdir if not have shell access - after changing permissions with sftp.setstat.

Before and after doing: chmod(sftp, "test.txt", 0o444) BEFORE:

-rw-******    ? 0        0              0B Oct 13 14:41 test.txt

AFTER:

sftp> ls -lh test.txt
-r--******    ? 0        0              0B Oct 13 14:41 test.txt

For your sake I also did it with pure sftp.setstat(): Before and after doing:

stat = sftp.stat("test.txt")
stat.permissions = LIBSSH2_SFTP_S_IRUSR |  LIBSSH2_SFTP_S_IRGRP | LIBSSH2_SFTP_S_IROTH
sftp.setstat("test.txt", stat)

BEFORE:

-rw-******    ? 0        0              0B Oct 13 14:41 test.txt

AFTER:

sftp> ls -lh test.txt
-r--******    ? 0        0              0B Oct 13 14:41 test.txt

If trying to go back to 666, with a file with only read access

stat = sftp.stat("test.txt")
stat.permissions = LIBSSH2_SFTP_S_IRUSR |  LIBSSH2_SFTP_S_IRGRP | LIBSSH2_SFTP_S_IROTH | LIBSSH2_SFTP_S_IWUSR |  LIBSSH2_SFTP_S_IWGRP | LIBSSH2_SFTP_S_IWOTH
sftp.setstat("test.txt", stat)

this happens:

Traceback (most recent call last):
  File "ssh2_test.py", line 73, in <module>
    sftp.setstat("test.txt", stat)
  File "ssh2\sftp.pyx", line 385, in ssh2.sftp.SFTP.setstat
  File "ssh2\utils.pyx", line 184, in ssh2.utils.handle_error_codes
ssh2.exceptions.SFTPProtocolError

For both these example, the same code to create the sftp is the same as I have shown before Why there is a bunch of *******, I do not know

[pkittenis]

Thanks for the code to reproduce.

So it looks like setstat requires that LIBSSH2_SFTP_ATTR_PERMISSIONS mask is set to attrs.flags when setting permissions otherwise the permissions may not be set correctly.

I assume this is also the case when setting size, time et al - #103 will probably also be resolved by setting correct attribute.

Eg:

rdonly_perms = LIBSSH2_SFTP_S_IRUSR |  LIBSSH2_SFTP_S_IRGRP | \
    LIBSSH2_SFTP_S_IROTH
rw_perms = LIBSSH2_SFTP_S_IRUSR |  LIBSSH2_SFTP_S_IRGRP | \
    LIBSSH2_SFTP_S_IROTH | LIBSSH2_SFTP_S_IWUSR |  \
    LIBSSH2_SFTP_S_IWGRP | LIBSSH2_SFTP_S_IWOTH

_mask = int('0666') if version_info <= (2,) else 0o666
os.chmod(remote_filename, _mask)
attrs = sftp.stat(remote_filename)
attrs.permissions = rdonly_perms
self.assertEqual(sftp.setstat(remote_filename, attrs), 0)
attrs = sftp.stat(remote_filename)
self.assertEqual(oct(attrs.permissions),'0o100444')
attrs.permissions = rw_perms
self.assertEqual(sftp.setstat(remote_filename, attrs), 0)
attrs = sftp.stat(remote_filename)
self.assertEqual(oct(attrs.permissions), '0o100666')

This fails. Setting flag before setstat sets permissions correctly.

self.assertEqual(sftp.setstat(remote_filename, attrs), 0)
attrs = sftp.stat(remote_filename)
self.assertEqual(oct(attrs.permissions),'0o100444')
attrs.permissions = rw_perms
attrs.flags = LIBSSH2_SFTP_ATTR_PERMISSIONS
self.assertEqual(sftp.setstat(remote_filename, attrs), 0)
attrs = sftp.stat(remote_filename)
self.assertEqual(oct(attrs.permissions), '0o100666')

Having to use attribute flags is not documented anywhere from what I can see and some permission flags work without LIBSSH2_SFTP_ATTR_PERMISSIONS - current integration tests do not use them and work for the read only flags that they test.

SFTP attribute flags are not implemented in the current release - will have to wait for next release to use them.

[pkittenis]

0.23.0 contains the LIBSSH2_SFTP_ATTR_* flags. To change permissions correctly set attrs.flags = LIBSSH2_SFTP_ATTR_PERMISSIONS per example above.

Thanks for raising. A complete example for setstat would be good to have if anyone wants to make a PR. There does not seem to be much libssh2 documentation on SFTP attributes.