search

ansible / pylibssh

Python bindings specific to Ansible use case for libssh https://www.libssh.org/
https://ansible-pylibssh.rtfd.io
GNU Lesser General Public License v2.1
59 stars 30 forks source link

sftp.get downloads only the last chunk of file #341

Open mumische opened 2 years ago

mumische commented 2 years ago
SUMMARY

I'm trying to download configuration file from remote router and getting only last chunk of the file.

ISSUE TYPE
PYLISSH and LIBSSH VERSION
0.4.0
OS / ENVIRONMENT

Ubuntu 22.04 LTS Python 3.10.4

STEPS TO REPRODUCE

Narrowing the problem I wrote a piece of code below. You need a file on remote host to reproduce it.

from pylibsshext.session import Session, AutoAddPolicy

hssh = Session()
p = AutoAddPolicy()
hssh.set_missing_host_key_policy(p)

HOST = 'hostname'
USER = 'username'
PASSWORD = 'password'
TIMEOUT = 30
PORT = 22

hssh.connect(
    host=HOST,
    user=USER,
    password=PASSWORD,
    timeout=TIMEOUT,
    port=PORT,
)
sftp = hssh.sftp()
sftp.get('m8.rsc', 'm8.rsc')
EXPECTED RESULTS

File 'm8.rsc' in current directory, size of 16K

ACTUAL RESULTS

File 'm8.rsc', containing last 223b of source file.

As I found out, the problem lays in line #103 of sftp.pyx

           with open(local_file, 'wb+') as f:

In python3, "w" parameter truncates the file, so during reading the source file new data overwrites old. Please check and consider changing open mode to 'ab'

Jakuje commented 6 months ago

The link to the actual code line:

https://github.com/ansible/pylibssh/blob/ba0797ccfefa9107a3be2d7e9726b07ef550b978/src/pylibsshext/sftp.pyx#L103

Reading through the python questions, this sounds like a common confusion and I think you are right that this should be ab instead of wb+:

https://stackoverflow.com/questions/1466000/difference-between-modes-a-a-w-w-and-r-in-built-in-open-function/30566011#30566011

Wondering why nobody noticed this earlier.

kucharskim commented 1 week ago

I hit this issue yesterday, while testing sftp feature with ansible-libssh and noticed this problem.

Jakuje commented 1 week ago

I hit this issue yesterday, while testing sftp feature with ansible-libssh and noticed this problem.

Yeah. Should have been fixed with #638. Unfortunately not yet merged :(