search

byteskeptical / sftpretty

Provides multi-threaded routines and high level protocol abstractions for a pretty quick & simple file transfer experience. Super duper close to a drop in replacement for pysftp.
https://docs.sftpretty.com/
BSD 3-Clause "New" or "Revised" License
33 stars 8 forks source link

Windows: No hostkey for host [localhost] found. #20

Closed dineshbvadhia closed 1 year ago

dineshbvadhia commented 1 year ago

cnopts = CnOpts(knownhosts='C:/Users/dines/.ssh/known_hosts') 

Traceback (most recent call last):
  File "C:\Users\dines\..test_sftp.py", line 48, in <module>
    main(sys.argv)
  File "C:\Users\dines\..test_sftp.py", line 41, in main
    with Connection(host='localhost', port=3373, username='admin', password='admin', cnopts=cnopts) as sftp:
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 140, in __init__
    self._start_transport(host, port, compress=compress)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 309, in _start_transport
    raise err
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 295, in _start_transport
    user_hostkey = self._cnopts.get_hostkey(host)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 98, in get_hostkey
    raise SSHException(f'No hostkey for host [{host}] found.')
paramiko.ssh_exception.SSHException: No hostkey for host [localhost] found.

Looking at: https://github.com/byteskeptical/sftpretty/blob/d8e2b07eaa050fa12cb32a37213e2f118248b173/sftpretty/__init__.py#L50 https://github.com/byteskeptical/sftpretty/blob/d8e2b07eaa050fa12cb32a37213e2f118248b173/sftpretty/__init__.py#L75

The code looks as if it should work or is something missing for Windows?

byteskeptical commented 1 year ago

Greetings Dinesh,

The message your are receiving is trying to say that you have never connected to this server using the hostname [localhost] (maybe you used the IP previously) which is why its host key is not found in your know_hosts. You can either manually connect to the server using the sftp localhost or ssh localhost commands to update your know_hosts file. Or you can disable host key verification by passing cnopts = CnOpts(knownhosts=None).

dineshbvadhia commented 1 year ago

The test_sftp.py code for cnopts with a knownhosts file and None is:

cnopts = CnOpts(knownhosts='C:/Users/dines/.ssh/known_hosts')       # 1. below
# cnopts = CnOpts(knownhosts=None)                                  # 2. below
with Connection(host='localhost', port=3373, username='admin', password='admin', cnopts=cnopts) as sftp:
    sftp.get('C:/Users/dines/source.txt', localpath='C:/Users/dines/data/destination.txt')

For both cases, the sftpserver is started fresh on localhost followed by an sftp localhost command to create a new knownhosts file each time. The debug ouput of the sftpserver is included. Running latest Windows 11.

  1. cnopts = CnOpts(knownhosts='C:/Users/dines/.ssh/known_hosts')
PS C:\Users\dines\...> sftp -P 3373 localhost
The authenticity of host '[localhost]:3373 ([127.0.0.1]:3373)' can't be established.
RSA key fingerprint is SHA256:91+xrZAGlWrRKg7yvUOmmbUB2Jio84Ue7yO9SjeLuRg.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])?
Warning: Permanently added '[localhost]:3373' (RSA) to the list of known hosts.
Connected to localhost.

PS C:\Users\dines\...> python test_sftp.py
[2023-01-28 13:32:11,741] INFO - [localhost] Host Key:
        Name: ssh-rsa
        Fingerprint: b'536c10077bb0490e53938ca85732b261'
        Size: 1024
Traceback (most recent call last):
  File "C:\Users\dines\...\test_sftp.py", line 50, in <module>
    main(sys.argv)
  File "C:\Users\dines\...\test_sftp.py", line 42, in main
    with Connection(host='localhost', port=3373, username='admin', password='admin', cnopts=cnopts) as sftp:
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 140, in __init__
    self._start_transport(host, port, compress=compress)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 309, in _start_transport
    raise err
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 295, in _start_transport
    user_hostkey = self._cnopts.get_hostkey(host)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 98, in get_hostkey
    raise SSHException(f'No hostkey for host [{host}] found.')
paramiko.ssh_exception.SSHException: No hostkey for host [localhost] found.

PS C:\Users\dines> sftpserver -l DEBUG
DEBUG:sftpserver.__main__:Serving C:\Users\dines over sftp at localhost:3373
DEBUG:sftpserver.__main__:Starting a new thread
INFO:paramiko.transport:Connected (version 2.0, client OpenSSH_for_Windows_8.6)
INFO:paramiko.transport:Auth rejected (none).
INFO:paramiko.transport:Auth granted (publickey).
DEBUG:sftpserver.__main__:1 active sessions
ERROR:paramiko.transport:Socket exception: An existing connection was forcibly closed by the remote host (10054)
DEBUG:sftpserver.__main__:Starting a new thread
INFO:paramiko.transport:Connected (version 2.0, client paramiko_2.12.0)
DEBUG:sftpserver.__main__:2 active sessions
  1. cnopts = CnOpts(knownhosts=None)
PS C:\Users\dines\...> sftp -P 3373 localhost
The authenticity of host '[localhost]:3373 ([127.0.0.1]:3373)' can't be established.
RSA key fingerprint is SHA256:rlA2aHp2rfRr3QsRVAtiCF6PZ9LVrtrur9wAWdRQk1E.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])?
Warning: Permanently added '[localhost]:3373' (RSA) to the list of known hosts.
Connected to localhost.

PS C:\Users\dines\...> python test_sftp.py
[2023-01-28 13:37:42,791] INFO - [localhost] Host Key:
        Name: ssh-rsa
        Fingerprint: b'1304f5dea9abeb5e17af4fcba54c4066'
        Size: 1024
Traceback (most recent call last):
  File "C:\Users\dines\...\test_sftp.py", line 50, in <module>
    main(sys.argv)
  File "C:\Users\dines\...\test_sftp.py", line 43, in main
    sftp.get('C:/Users/dines/source.txt', localpath='C:/Users/dines/data/destination.txt')
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 366, in get
    _get(self, remotefile, localpath=localpath, callback=callback,
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 359, in _get
    channel.get(remotefile, localpath=localpath,
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\sftp_client.py", line 811, in get
    size = self.getfo(remotepath, fl, callback, prefetch)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\sftp_client.py", line 782, in getfo
    file_size = self.stat(remotepath).st_size
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\sftp_client.py", line 493, in stat
    t, msg = self._request(CMD_STAT, path)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\sftp_client.py", line 822, in _request
    return self._read_response(num)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\sftp_client.py", line 874, in _read_response
    self._convert_status(msg)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\sftp_client.py", line 907, in _convert_status
    raise IOError(text)
OSError: Failure

PS C:\Users\dines> sftpserver -l DEBUG
DEBUG:sftpserver.__main__:Serving C:\Users\dines over sftp at localhost:3373
DEBUG:sftpserver.__main__:Starting a new thread
INFO:paramiko.transport:Connected (version 2.0, client OpenSSH_for_Windows_8.6)
INFO:paramiko.transport:Auth rejected (none).
INFO:paramiko.transport:Auth granted (publickey).
DEBUG:sftpserver.__main__:1 active sessions
ERROR:paramiko.transport:Socket exception: An existing connection was forcibly closed by the remote host (10054)
DEBUG:sftpserver.__main__:Starting a new thread
INFO:paramiko.transport:Connected (version 2.0, client paramiko_2.12.0)
INFO:paramiko.transport:Auth granted (password).
DEBUG:sftpserver.__main__:2 active sessions
ERROR:paramiko.transport:Socket exception: An established connection was aborted by the software in your host machine (10053)

I did expect both 1 and 2 to work :(

byteskeptical commented 1 year ago

I'm a bit confused. The original issue was with not finding a matching host key in your know_hosts file. Seems that is working now. I'm unsure what the new problem is though? You say your running this on Windows 11 but have an OpenSSH build for Windows 8.6? Your using 1024 RSA host keys that have been deprecated since OpenSSH 8.8 (08/20/2021). Perhaps your modifying your options for this but I wouldn't know since you don't share this. It does explain why example #2 works and #1 does not. You can either try using another host key type like ed25519 or pass HostKeyAlgorithms=+ssh-rsa and PubkeyAcceptedKeyTypes=+ssh-rsa to use existing keys.

dineshbvadhia commented 1 year ago

It is very confusing. I think "OpenSSH_for_Windows_8.6" means OpenSSH for Windows v8.6 - the 8.6 could refer to the version of OpenSSH.

byteskeptical commented 1 year ago

I see, that seems more likely. Can you try running the above examples but with the following CnOpts for both:

cnopts.kex = ('ssh-rsa',) cnopts.key_types = ('ssh-rsa',)

That should produce the expected behavior. If so then the problem is as stated above, a version mis-match between client and server where the server accepts these old deprecated keys but the client is rejecting them by default due to new requirements.

dineshbvadhia commented 1 year ago

Both

1.
cnopts = CnOpts(knownhosts='C:/Users/dines/.ssh/known_hosts',)
cnopts.kex = ('ssh-rsa',)
cnopts.key_types = ('ssh-rsa',)

2. 
cnopts = CnOpts(knownhosts=None,)
cnopts.kex = ('ssh-rsa',)
cnopts.key_types = ('ssh-rsa',)

Give the same error:

PS C:\Users\dines\> sftp -P 3373 localhost
The authenticity of host '[localhost]:3373 ([127.0.0.1]:3373)' can't be established.
RSA key fingerprint is SHA256:UJte06LQo9BT4ZpaQDd6wSWtcALB31ShZzqTJVtBGmA.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])?
Warning: Permanently added '[localhost]:3373' (RSA) to the list of known hosts.
Connected to localhost.
PS C:\Users\dines\> python test_sftp.py
Traceback (most recent call last):
  File "C:\Users\dines\\test_sftp.py", line 43, in <module>
    main()
  File "C:\Users\dines\\test_sftp.py", line 35, in main
    with Connection(host='localhost', port=3373, username='admin', password='admin', cnopts=cnopts) as sftp:
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 140, in __init__
    self._start_transport(host, port, compress=compress)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 309, in _start_transport
    raise err
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 277, in _start_transport
    self._transport.get_security_options().kex = kex
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\transport.py", line 3078, in kex
    self._set("_preferred_kex", "_kex_info", x)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\transport.py", line 3041, in _set
    raise ValueError("unknown cipher")
ValueError: unknown cipher
PS C:\Users\dines> sftpserver -l DEBUG
DEBUG:sftpserver.__main__:Serving C:\Users\dines over sftp at localhost:3373
DEBUG:sftpserver.__main__:Starting a new thread
INFO:paramiko.transport:Connected (version 2.0, client OpenSSH_for_Windows_8.6)
INFO:paramiko.transport:Auth rejected (none).
INFO:paramiko.transport:Auth granted (publickey).
DEBUG:sftpserver.__main__:1 active sessions
ERROR:paramiko.transport:Socket exception: An existing connection was forcibly closed by the remote host (10054)
DEBUG:sftpserver.__main__:Starting a new thread
ERROR:paramiko.transport:Exception (server): Error reading SSH protocol banner[WinError 10054] An existing connection was forcibly closed by the remote host
ERROR:paramiko.transport:Traceback (most recent call last):
ERROR:paramiko.transport:  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\transport.py", line 2268, in _check_banner
ERROR:paramiko.transport:    buf = self.packetizer.readline(timeout)
ERROR:paramiko.transport:  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\packet.py", line 374, in readline
ERROR:paramiko.transport:    buf += self._read_timeout(timeout)
ERROR:paramiko.transport:  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\packet.py", line 601, in _read_timeout
ERROR:paramiko.transport:    x = self.__socket.recv(128)
ERROR:paramiko.transport:ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host
ERROR:paramiko.transport:
ERROR:paramiko.transport:During handling of the above exception, another exception occurred:
ERROR:paramiko.transport:
ERROR:paramiko.transport:Traceback (most recent call last):
ERROR:paramiko.transport:  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\transport.py", line 2091, in run
ERROR:paramiko.transport:    self._check_banner()
ERROR:paramiko.transport:  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\transport.py", line 2272, in _check_banner
ERROR:paramiko.transport:    raise SSHException(
ERROR:paramiko.transport:paramiko.ssh_exception.SSHException: Error reading SSH protocol banner[WinError 10054] An existing connection was forcibly closed by the remote host
ERROR:paramiko.transport:
Traceback (most recent call last):
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\transport.py", line 2268, in _check_banner
    buf = self.packetizer.readline(timeout)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\packet.py", line 374, in readline
    buf += self._read_timeout(timeout)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\packet.py", line 601, in _read_timeout
    x = self.__socket.recv(128)
ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\Scripts\sftpserver.exe\__main__.py", line 7, in <module>
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpserver\__main__.py", line 174, in main
    start_server(args.host, args.port, args.root, args.keyfile, password, args.level, args.mode)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpserver\__main__.py", line 125, in start_server
    transport = setup_transport(connection)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpserver\__main__.py", line 74, in setup_transport
    transport.start_server(server=ssh_server)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\transport.py", line 770, in start_server
    raise e
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\transport.py", line 2091, in run
    self._check_banner()
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\transport.py", line 2272, in _check_banner
    raise SSHException(
paramiko.ssh_exception.SSHException: Error reading SSH protocol banner[WinError 10054] An existing connection was forcibly closed by the remote host
byteskeptical commented 1 year ago

What's the output ssh -Q kex and ssh -Q key? Seems ssh-rsa is not a valid kex option, you can try just cnopts.key = ('ssh-rsa',) which may be enough. Either way it seems the issue is with support for RSA host keys of the size your using. You can either play around with CnOpts.kex and CnOpts.key further to get a connection object that makes both ends happy or switch to a host key alorgithm/size that isn't deprecated. To be clear the issue here is your use of 1024 bit RSA host key for your server. Try bumping that to at least 2048 and you won't need to pass any additional CnOpts other than the knownhosts.

dineshbvadhia commented 1 year ago

Bumping key size to 4096 didn't do it either :(

Results below are with: cnopts = CnOpts(knownhosts='C:/Users/dines/.ssh/known_hosts',) but also tried with: cnopts = CnOpts(knownhosts=None,)

Can you recommend a different test sftp server I can try with?

PS C:\Users\dines\> sftp -P 3373 localhost
The authenticity of host '[localhost]:3373 ([127.0.0.1]:3373)' can't be established.
RSA key fingerprint is SHA256:jQy4Prfn/J5Ch/oRzM8V9YiPJhSoOdQGVVgWyAV0Ylg.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])?
Warning: Permanently added '[localhost]:3373' (RSA) to the list of known hosts.
Connected to localhost.
PS C:\Users\dines\> python test_sftp.py
[2023-02-07 10:48:25,493] INFO - [localhost] Host Key:
        Name: ssh-rsa
        Fingerprint: b'a6acab1602c4bf2fa208265a64b61b3d'
        Size: 4096
Traceback (most recent call last):
  File "C:\Users\dines\\test_sftp.py", line 47, in <module>
    main()
  File "C:\Users\dines\\test_sftp.py", line 39, in main
    with Connection(host='localhost', port=3373, username='admin', password='admin', cnopts=cnopts) as sftp:
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 140, in __init__
    self._start_transport(host, port, compress=compress)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 309, in _start_transport
    raise err
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 295, in _start_transport
    user_hostkey = self._cnopts.get_hostkey(host)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 98, in get_hostkey
    raise SSHException(f'No hostkey for host [{host}] found.')
paramiko.ssh_exception.SSHException: No hostkey for host [localhost] found.
PS C:\Users\dines> sftpserver -l DEBUG
DEBUG:sftpserver.__main__:Serving C:\Users\dines over sftp at localhost:3373
DEBUG:sftpserver.__main__:Starting a new thread
INFO:paramiko.transport:Connected (version 2.0, client OpenSSH_for_Windows_8.6)
INFO:paramiko.transport:Auth rejected (none).
INFO:paramiko.transport:Auth granted (publickey).
DEBUG:sftpserver.__main__:1 active sessions
ERROR:paramiko.transport:Socket exception: An existing connection was forcibly closed by the remote host (10054)
DEBUG:sftpserver.__main__:Starting a new thread
INFO:paramiko.transport:Connected (version 2.0, client paramiko_3.0.0)
DEBUG:sftpserver.__main__:2 active sessions
> ssh -Q kex
diffie-hellman-group1-sha1
diffie-hellman-group14-sha1
diffie-hellman-group14-sha256
diffie-hellman-group16-sha512
diffie-hellman-group18-sha512
diffie-hellman-group-exchange-sha1
diffie-hellman-group-exchange-sha256
ecdh-sha2-nistp256
ecdh-sha2-nistp384
ecdh-sha2-nistp521
curve25519-sha256
curve25519-sha256@libssh.org
> ssh -Q key
ssh-ed25519
ssh-ed25519-cert-v01@openssh.com
sk-ssh-ed25519@openssh.com
sk-ssh-ed25519-cert-v01@openssh.com
ssh-rsa
ssh-dss
ecdsa-sha2-nistp256
ecdsa-sha2-nistp384
ecdsa-sha2-nistp521
sk-ecdsa-sha2-nistp256@openssh.com
ssh-rsa-cert-v01@openssh.com
ssh-dss-cert-v01@openssh.com
ecdsa-sha2-nistp256-cert-v01@openssh.com
ecdsa-sha2-nistp384-cert-v01@openssh.com
ecdsa-sha2-nistp521-cert-v01@openssh.com
sk-ecdsa-sha2-nistp256-cert-v01@openssh.com
byteskeptical commented 1 year ago

I don't mind attempting to help troubleshoot but your not giving me much to work with and clearly obfuscating the parts that you are. I need to know the host key your using so I can verify it's fingerprint, I need to know what's in the test_sftp.py script. I need the whole log output and traceback with debug enabled cnopts.log_level = 'debug' as well as the contents of your known_hosts file at C:/Users/dines/.ssh/known_hosts. Also your syntax for passing in the knownhosts option is wrong in your last message it was correct previously cnopts = CnOpts(knownhosts='C:/Users/dines/.ssh/known_hosts'). Lastly can you try adding a print of the knownhosts variable right above line 73 in the init.py file.

dineshbvadhia commented 1 year ago

Oh, I'm not deliberately obfuscating anything except possibly through ignorance. I'll start again from the top and let you know.

byteskeptical commented 1 year ago

I believe you, I only meant that at this point I need whole output as I suspect we are missing something simple that is causing this behavior.

dineshbvadhia commented 1 year ago

Here is the requested actions and output:

Generate key ('server_rsa')

PS C:\Users\dines> ssh-keygen -t rsa -b 4096
Generating public/private rsa key pair.
Enter file in which to save the key (C:\Users\dines/.ssh/id_rsa): c:\users\dines\.ssh\server_rsa
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in c:\users\dines\.ssh\server_rsa
Your public key has been saved in c:\users\dines\.ssh\server_rsa.pub
The key fingerprint is:
SHA256:RoEedrEjOHQJtq36SCEXrHJogl3a2TAGqvzBjljLRm0 dines@T2
The key's randomart image is:
+---[RSA 4096]----+
|  . +..oo.       |
| o + == .o       |
|. o Oooo+        |
|++.B B.o .       |
|OoOoE . S        |
|=B+=.  .         |
|..*o             |
| o o             |
|  . .            |
+----[SHA256]-----+

Show generated key

-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAACFwAAAAdzc2gtcn
NhAAAAAwEAAQAAAgEAzRHqtPgQ8bhogR5SX7k/mdGEtd6X/+w9oLdfawToAUIVjf8TntnV
TVUmngdNWJdk9gvwctmHwjKAfQH8RcbKNtppWHPBgfuHJzSv9yd378MMPIRt6HnQhZ97rR
/IbTxJ1jPEckJvii5tWDqNxiOqn/LCZmxGBDqJet/pBVsF/7Ij6xTImIGoYe6U83+/NnQh
EXj9+3/pbHtq/TPKtYWN/GMHzVw0s+BBx9I/x+CuJXjzTB1Pggl15LN7iOHtezbnlUggc9
kiiZZPPn/qN/HJlDP16egZNaES8bSCIo/mAMN6Sg+NBSvHAVCtzPOT5oToiDkgCow7ST67
/M5pkwKrFSwauyMQsGktx11Tk5KD3RtRXdj0L/+Ex8NlgdRStPv0nTqfOJZZEz/dwNEcey
1OEUeqrivQurkgi9R5cXDCxxaS4s9faHW1MNydHsLPxYRm1uP1jyDZ7rZwMznXWezCQg86
ChoHkiiSWPXkBVM7E9354kMfWONHs1o19rKGI7TDDycAB7umMCVSs+x8KpQSUs2CWjtpeC
usgTS+OsPk88uu3umMy07Q6XOiSTcudYLabZPE0bdI3u1p8TCp7MCvs25bjz1IOU1NCETX
8nwU4WCxUJbyDpFaG/vy6RbDqcJSx6kbxr0DEC0m/kV0cTKLmr4kBAkKzVG9EYt2Py2dhU
sAAAdAu8Pdy7vD3csAAAAHc3NoLXJzYQAAAgEAzRHqtPgQ8bhogR5SX7k/mdGEtd6X/+w9
oLdfawToAUIVjf8TntnVTVUmngdNWJdk9gvwctmHwjKAfQH8RcbKNtppWHPBgfuHJzSv9y
d378MMPIRt6HnQhZ97rR/IbTxJ1jPEckJvii5tWDqNxiOqn/LCZmxGBDqJet/pBVsF/7Ij
6xTImIGoYe6U83+/NnQhEXj9+3/pbHtq/TPKtYWN/GMHzVw0s+BBx9I/x+CuJXjzTB1Pgg
l15LN7iOHtezbnlUggc9kiiZZPPn/qN/HJlDP16egZNaES8bSCIo/mAMN6Sg+NBSvHAVCt
zPOT5oToiDkgCow7ST67/M5pkwKrFSwauyMQsGktx11Tk5KD3RtRXdj0L/+Ex8NlgdRStP
v0nTqfOJZZEz/dwNEcey1OEUeqrivQurkgi9R5cXDCxxaS4s9faHW1MNydHsLPxYRm1uP1
jyDZ7rZwMznXWezCQg86ChoHkiiSWPXkBVM7E9354kMfWONHs1o19rKGI7TDDycAB7umMC
VSs+x8KpQSUs2CWjtpeCusgTS+OsPk88uu3umMy07Q6XOiSTcudYLabZPE0bdI3u1p8TCp
7MCvs25bjz1IOU1NCETX8nwU4WCxUJbyDpFaG/vy6RbDqcJSx6kbxr0DEC0m/kV0cTKLmr
4kBAkKzVG9EYt2Py2dhUsAAAADAQABAAACAGYB+5Z5rzQOXy2bP66OH60MI8+0IjgT87P2
g2hg0GePz9kKgWALS6dkLAl/6GGhLX+XAMygmH9pT7DfQ/YRpJUUoEJ3klJuJBTEiKOQ5o
BFrdiMp3902N0fMA42OSTdUH1U0utjUMiTv6Cbv9pH86AMydbZFRsVYz3UE72uQZWnr2ZI
uF/kXzYXz9AwQ5V6vthKL0zjm+Fyow3ArZ0OCkr5l+/3Zd/0NcSIOWHbycSqHZxHGD2/Jv
3Uqia0+8TG0vvXHQ2DfUKjmlekmYWqqHMoRbNM2P1sWlgRSoqWufq3QiWFBskDtcJZt5qz
r2aDxoe8osucZgFRS4G9y5ZtRVwlqJTzMzttaweHsC1W/EEZxqNAHzh4DW83j9Ht5/S85v
KlpjJxwTWI95os8h5Cf8fXaaxhU/xhMT5iVYBna4HmIWskb+3rt/GzygL3Mf5zuQ5kH9IQ
qJ0fGjxGIrTJHBq21ZTuE1/VcIGzOYaVENYq2C7XFpfhPIuCIpXGtHLNnGWS7p2dxf/C10
ZGf2XBZzm0nKpRL0y7ildh2YIJgKIgnUQW3LbmkhhuO+80K62PQd/5WCpnpByPo0s1k6oU
ik01Fz/coDqFy3wikUrRiGBFeYUH0rpy/V33CmwJ9QBwJVabRXDc9po+pL4XizeWgV3JHM
Oexw0/EF1/LlDFV+MBAAABAESAJYgKWqk0BVQhVPRk3QnWPkL1x3BHZCYwdHCEBOZdNUYp
ZpUH091UQ4nhkDBrBLPVDr6DT5gLSgLALzgHFdm6kvO7oFVUjU0ZswPD0iapXbDsBOLRiX
CKP/KEIBGvFeCo0DJx2r8JToZBj1Jljt1BXQLNoRZdjPs/VReAWXTdhJerDPAqIZ8e3Bgs
/j0TnWnT+UYHWvryl/ef7d9lIlhkSXt2TId47t2MM/CGzdNNR7qoAgQ0v3eUKKS47DB9+B
gDVShv74fZMdjbBySgqy8dhH1KKUoLFzHDTEP6rJgYGTLoQ5RtV72Zhtum6vvmSyM2a8Xk
/0zhri41OuELFDwAAAEBAP3DCcGv7N/B9iqaCmkozCWg2SnDKYRy64+iZrkyQDHXaTDaF+
0Ey2042Mj8loKlZOI2+C022d1rE5eq9MmWved0f1Zxm1cAIGrAz7yLn3YT2//mkFGyi5kQ
GmFiPyL3Z8UpIN3W/o5ijhYXH7pf+nyoOnZqhaTE7+I8rImgVVBE+U8sYqs7KSwlhdE9/6
Vl/j4WqYvwYTC3wxYDyPX+IXEkY/X4pF9fNDy1dG0N/+XXzS+yn1O9zoEPZPkqo6RdYI1B
7mcZk4h57n/b+OlGvqzgPWcyiXFqEUKHHGvFWQhpt1SgMhG5MRjiAANy2UndYu2NN2hqs+
wjKkEREnbWvVsAAAEBAM7g8Exk26fGvqTwBp+zNHg8YjRq8VC+gEEWhZt1Hp9NpIQ0Iitn
srqbFNnak0aZQ31gFKTTUNRQhJXbmGG5D6RRlB0PFBSL/7RsPm4SeZB+0DGoa80/hz1bW9
+woxUL1/Ebj69XGllqGR8JcXM64J92hNA+miPm/YbQlOhsvHjaaKXZq+CVAwJLExxEKkw9
LHMZx1F6fPsm2gr+w7NeqGD0HC7wdI7AzwgmteFZLJdMQpJ+2X9CaKip/swkotGnj0GPUW
Yrc0aBSojiPQWuhZnrgN0DabB+I1Dh0Wh055Wd4CQD30hpKbvC2fnQONh3BstKv1gbk2mD
0S5Ip2B0KtEAAAAIZGluZXNAVDIBAgM=
-----END OPENSSH PRIVATE KEY-----

Start sftpserver (https://github.com/lonetwin/sftpserver) on 127.0.0.1 and port 3373 in debug mode

> sftpserver -k C:\Users\dines\.ssh\server_rsa -l DEBUG
DEBUG:sftpserver.__main__:Serving C:\Users\dines over sftp at localhost:3373
DEBUG:sftpserver.__main__:Starting a new thread
INFO:paramiko.transport:Connected (version 2.0, client OpenSSH_for_Windows_8.6)
INFO:paramiko.transport:Auth rejected (none).
INFO:paramiko.transport:Auth granted (publickey).
DEBUG:sftpserver.__main__:1 active sessions
ERROR:paramiko.transport:Socket exception: An existing connection was forcibly closed by the remote host (10054)
DEBUG:sftpserver.__main__:Starting a new thread
INFO:paramiko.transport:Connected (version 2.0, client paramiko_3.0.0)
DEBUG:sftpserver.__main__:2 active sessions
DEBUG:sftpserver.__main__:Starting a new thread
INFO:paramiko.transport:Connected (version 2.0, client paramiko_3.0.0)
DEBUG:sftpserver.__main__:3 active sessions

Create known_hosts file

> sftp -P 3373 127.0.0.1
PS C:\Users\dines> sftp -P 3373 127.0.0.1
The authenticity of host '[127.0.0.1]:3373 ([127.0.0.1]:3373)' can't be established.
RSA key fingerprint is SHA256:RoEedrEjOHQJtq36SCEXrHJogl3a2TAGqvzBjljLRm0.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])?
Warning: Permanently added '[127.0.0.1]:3373' (RSA) to the list of known hosts.
Connected to 127.0.0.1.

Show known_hosts file

[127.0.0.1]:3373 ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDNEeq0+BDxuGiBHlJfuT+Z0YS13pf/7D2gt19rBOgBQhWN/xOe2dVNVSaeB01Yl2T2C/By2YfCMoB9AfxFxso22mlYc8GB+4cnNK/3J3fvwww8hG3oedCFn3utH8htPEnWM8RyQm+KLm1YOo3GI6qf8sJmbEYEOol63+kFWwX/siPrFMiYgahh7pTzf782dCEReP37f+lse2r9M8q1hY38YwfNXDSz4EHH0j/H4K4lePNMHU+CCXXks3uI4e17NueVSCBz2SKJlk8+f+o38cmUM/Xp6Bk1oRLxtIIij+YAw3pKD40FK8cBUK3M85PmhOiIOSAKjDtJPrv8zmmTAqsVLBq7IxCwaS3HXVOTkoPdG1Fd2PQv/4THw2WB1FK0+/SdOp84llkTP93A0Rx7LU4RR6quK9C6uSCL1HlxcMLHFpLiz19odbUw3J0ews/FhGbW4/WPINnutnAzOddZ7MJCDzoKGgeSKJJY9eQFUzsT3fniQx9Y40ezWjX2soYjtMMPJwAHu6YwJVKz7HwqlBJSzYJaO2l4K6yBNL46w+Tzy67e6YzLTtDpc6JJNy51gtptk8TRt0je7WnxMKnswK+zbluPPUg5TU0IRNfyfBThYLFQlvIOkVob+/LpFsOpwlLHqRvGvQMQLSb+RXRxMouaviQECQrNUb0Ri3Y/LZ2FSw==

The test_sftp program source:

from sftpretty import CnOpts, Connection

def main():
    cnopts = CnOpts(knownhosts='C:/Users/dines/.ssh/known_hosts')
    # cnopts = CnOpts(knownhosts=None)
    cnopts.log_level = 'debug'
    with Connection(host='localhost', port=3373, username='admin', password='admin', cnopts=cnopts) as sftp:
        sftp.get('C:/Users/dines/source.txt', localpath='C:/Users/dines/data/destination.txt')
    return

if __name__ == '__main__':
    main()

Run test_sftp

PS C:\Users\dines\Thingy\thingy-engine> python test_sftp.py
[2023-02-14 17:15:46,790] DEBUG - Disabled Algorithms: [{}]
[2023-02-14 17:15:46,790] DEBUG - Ciphers: [('aes256-ctr', 'aes192-ctr', 'aes128-ctr', 'aes256-cbc', 'aes192-cbc', 'aes128-cbc', '3des-cbc')]
[2023-02-14 17:15:46,791] DEBUG - Compression: [('none',)]
[2023-02-14 17:15:46,792] DEBUG - MACs: [('hmac-sha2-512', 'hmac-sha2-256', 'hmac-sha2-512-etm@openssh.com', 'hmac-sha2-256-etm@openssh.com', 'hmac-sha1', 'hmac-md5')]
[2023-02-14 17:15:46,792] DEBUG - KEX: [('ecdh-sha2-nistp521', 'ecdh-sha2-nistp384', 'ecdh-sha2-nistp256', 'diffie-hellman-group16-sha512', 'diffie-hellman-group-exchange-sha256', 'diffie-hellman-group-exchange-sha1')]
[2023-02-14 17:15:46,792] DEBUG - Public Key Types: [('ssh-ed25519', 'ecdsa-sha2-nistp521', 'ecdsa-sha2-nistp384', 'ecdsa-sha2-nistp256', 'rsa-sha2-512', 'rsa-sha2-256', 'ssh-rsa', 'ssh-dss')]
[2023-02-14 17:15:46,809] INFO - [127.0.0.1] Host Key:
        Name: ssh-rsa
        Fingerprint: b'ab80bef9f1ba7fbcc5378c6d774464ec'
        Size: 4096
Traceback (most recent call last):
  File "C:\Users\dines\Thingy\thingy-engine\test_sftp.py", line 34, in <module>
    main()
  File "C:\Users\dines\Thingy\thingy-engine\test_sftp.py", line 28, in main
    with Connection(host='127.0.0.1', port=3373, username='admin', password='admin', cnopts=cnopts) as sftp:
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 140, in __init__
    self._start_transport(host, port, compress=compress)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 309, in _start_transport
    raise err
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 295, in _start_transport
    user_hostkey = self._cnopts.get_hostkey(host)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 98, in get_hostkey
    raise SSHException(f'No hostkey for host [{host}] found.')
paramiko.ssh_exception.SSHException: No hostkey for host [127.0.0.1] found.

Run test_sftp with L73 print of knownhosts

PS C:\Users\dines\Thingy\thingy-engine> python test_sftp.py

Line73 knownhosts: C:/Users/dines/.ssh/known_hosts

[2023-02-14 17:17:20,866] DEBUG - Disabled Algorithms: [{}]
[2023-02-14 17:17:20,866] DEBUG - Ciphers: [('aes256-ctr', 'aes192-ctr', 'aes128-ctr', 'aes256-cbc', 'aes192-cbc', 'aes128-cbc', '3des-cbc')]
[2023-02-14 17:17:20,866] DEBUG - Compression: [('none',)]
[2023-02-14 17:17:20,866] DEBUG - MACs: [('hmac-sha2-512', 'hmac-sha2-256', 'hmac-sha2-512-etm@openssh.com', 'hmac-sha2-256-etm@openssh.com', 'hmac-sha1', 'hmac-md5')]
[2023-02-14 17:17:20,866] DEBUG - KEX: [('ecdh-sha2-nistp521', 'ecdh-sha2-nistp384', 'ecdh-sha2-nistp256', 'diffie-hellman-group16-sha512', 'diffie-hellman-group-exchange-sha256', 'diffie-hellman-group-exchange-sha1')]
[2023-02-14 17:17:20,866] DEBUG - Public Key Types: [('ssh-ed25519', 'ecdsa-sha2-nistp521', 'ecdsa-sha2-nistp384', 'ecdsa-sha2-nistp256', 'rsa-sha2-512', 'rsa-sha2-256', 'ssh-rsa', 'ssh-dss')]
[2023-02-14 17:17:20,878] INFO - [127.0.0.1] Host Key:
        Name: ssh-rsa
        Fingerprint: b'ab80bef9f1ba7fbcc5378c6d774464ec'
        Size: 4096
Traceback (most recent call last):
  File "C:\Users\dines\Thingy\thingy-engine\test_sftp.py", line 34, in <module>
    main()
  File "C:\Users\dines\Thingy\thingy-engine\test_sftp.py", line 28, in main
    with Connection(host='127.0.0.1', port=3373, username='admin', password='admin', cnopts=cnopts) as sftp:
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 141, in __init__
    self._start_transport(host, port, compress=compress)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 310, in _start_transport
    raise err
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 296, in _start_transport
    user_hostkey = self._cnopts.get_hostkey(host)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 99, in get_hostkey
    raise SSHException(f'No hostkey for host [{host}] found.')
paramiko.ssh_exception.SSHException: No hostkey for host [127.0.0.1] found.

Additionally, also ran the sequence of statements on another identical laptop and got the same set of results.

dineshbvadhia commented 1 year ago

Hello. Any joy?

I followed the example at the bottom of https://github.com/lonetwin/sftpserver and added the sftp.get and all statements worked. Does this mean the problem is with sftpretty?

>>> import paramiko
>>> pkey = paramiko.RSAKey.from_private_key_file('..\..\.ssh\id_rsa')
>>> transport = paramiko.Transport(('localhost', 3373))
>>> transport.connect(username='admin', password='admin', pkey=pkey)
>>> sftp = paramiko.SFTPClient.from_transport(transport)
>>> sftp.listdir('..\..\.ssh')
['id_rsa', 'id_rsa.pub', 'key.pem', 'known_hosts', 'known_hosts.old']
>>> sftp.get('..\..\source.txt', localpath='..\..\data/destination.txt')
dineshbvadhia commented 1 year ago

Digging in init.py:

L50

> print(Path('~/.ssh/known_hosts').expanduser().as_posix())
C:/Users/dines/.ssh/known_hosts

L71

> print(knownhosts)
../../.ssh/known_hosts

L73

> print(Path(knownhosts).resolve().as_posix())
C:/Users/dines/.ssh/known_hosts

Maybe the problem lies here?

dineshbvadhia commented 1 year ago

I've noticed with:

    # cannot find known_hosts
    cnopts = CnOpts(knownhosts='../../.ssh/known_hosts')

    # file name transfers but not the content (see traceback below)
    cnopts = CnOpts(knownhosts=None)

test_sftp.py

from sftpretty import CnOpts, Connection

def main():
    # cnopts = CnOpts(knownhosts='../../.ssh/known_hosts')
    cnopts = CnOpts(knownhosts=None)
    cnopts.log_level = 'debug'

    with Connection('localhost', port=3373, cnopts=cnopts, username='admin', password='admin') as sftp:        
        sftp.get('../../source.txt', localpath='../../data/destination.txt')
    return

if __name__ == '__main__':
    main()
PS C:\Users\dines\> python test_sftp.py
[2023-02-26 17:38:31,993] DEBUG - Disabled Algorithms: [{}]
[2023-02-26 17:38:31,993] DEBUG - Ciphers: [('aes256-ctr', 'aes192-ctr', 'aes128-ctr', 'aes256-cbc', 'aes192-cbc', 'aes128-cbc', '3des-cbc')]
[2023-02-26 17:38:31,993] DEBUG - Compression: [('none',)]
[2023-02-26 17:38:31,997] DEBUG - MACs: [('hmac-sha2-512', 'hmac-sha2-256', 'hmac-sha2-512-etm@openssh.com', 'hmac-sha2-256-etm@openssh.com', 'hmac-sha1', 'hmac-md5')]
[2023-02-26 17:38:31,997] DEBUG - KEX: [('ecdh-sha2-nistp521', 'ecdh-sha2-nistp384', 'ecdh-sha2-nistp256', 'diffie-hellman-group16-sha512', 'diffie-hellman-group-exchange-sha256', 'diffie-hellman-group-exchange-sha1')]
[2023-02-26 17:38:31,997] DEBUG - Public Key Types: [('ssh-ed25519', 'ecdsa-sha2-nistp521', 'ecdsa-sha2-nistp384', 'ecdsa-sha2-nistp256', 'rsa-sha2-512', 'rsa-sha2-256', 'ssh-rsa', 'ssh-dss')]
[2023-02-26 17:38:32,027] INFO - [localhost] Host Key:
        Name: ssh-rsa
        Fingerprint: b'2b52128ac64a616190d8e3dafbf2e672'
        Size: 4096
[2023-02-26 17:38:32,029] DEBUG - Retry: [DISABLED]
[2023-02-26 17:38:32,029] DEBUG - Channel Name: [2b18dfb2547a4200bc42d71e589b9e41]
Traceback (most recent call last):
  File "C:\Users\dines\Thingy\test_sftp.py", line 13, in <module>
    main()
  File "C:\Users\dines\Thingy\test_sftp.py", line 9, in main
    sftp.get('../../source.txt', localpath='../../data/destination.txt')
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 366, in get
    _get(self, remotefile, localpath=localpath, callback=callback,
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\sftpretty\__init__.py", line 359, in _get
    channel.get(remotefile, localpath=localpath,
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\sftp_client.py", line 811, in get
    size = self.getfo(remotepath, fl, callback, prefetch)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\sftp_client.py", line 782, in getfo
    file_size = self.stat(remotepath).st_size
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\sftp_client.py", line 493, in stat
    t, msg = self._request(CMD_STAT, path)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\sftp_client.py", line 822, in _request
    return self._read_response(num)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\sftp_client.py", line 874, in _read_response
    self._convert_status(msg)
  File "C:\Users\dines\AppData\Local\Programs\Python\Python310\lib\site-packages\paramiko\sftp_client.py", line 903, in _convert_status
    raise IOError(errno.ENOENT, text)
FileNotFoundError: [Errno 2] No such file
byteskeptical commented 1 year ago

I see a few issues but all are user errors. When you create your known_hosts file you use 127.0.0.1 which shows in the content of the created file in the next comment. In your script you are passing in localhost in your connection string. These are two seperate entries as you can see in the sftpserver.pub example file in the repo. Thus the error message about not finding the ssh host key. Again I'm not sure what the issue is in your most recent group of messages. You're relative remote path doesn't exist which the error message is pretty clear about. If you have another issue I'm missing here please open a new specific issue with simplified, reproducible, library code. I would also suggest that you review the examples in the README since the way you're passing in the host parameter in the Connection object is not correct.