search

kolban-google / sftp-gcs

An implementation of an SFTP to Google Cloud Storage bridge.
Apache License 2.0
90 stars 36 forks source link

Issue with Filezilla #2

Open kolban-google opened 3 years ago

kolban-google commented 3 years ago

A slack post/email was received from a user:

As suggested in your reply, we can exchange by mail. First of all, thank you for your work on sftp-gcs that's exactly what I need. We need files from Partners to import in Biquery. They can not access GCS and upload their CSVs in our bucket, they only can do that with SFTP. We can't access their environment. As you can imagine your sftp-gcs is more than welcome :)

If I use no authentication, I can copy files in from a console or with software like Transmit (mac - https://panic.com/transmit/). But Filezilla on PC...no way... But what I need is through Private/Public keys.

I made my keys (PuTTYGen).

I started the server like that 

node sftp-gcs.js --debug=debug --bucket=gs://ipb-ftp-storage \
  --port=9022 --public-key-file=keys/ftppublic.key

And tried to connect with my mac (transmit) & with Filezilla PC. In both cases the same error.

Note: There are images in the emails that are private and hence will not be shared.

kolban-google commented 3 years ago

This puzzle has a few dimensions to it. The first was that we had an apparent failure to process incoming connection requests from FileZilla that were requesting key exchange (as opposed to none or password).

When an sftp client connects to an sftp server, the client is supposed to declare its authentication style desired ... this can be either:

In our story, the user was passing in a publickey and failing. The underlying error showed that despite desiring to pass a public key, the FileZilla is saying that it wants "none". But since we had configured our gcs-sftp server to expect a publickey, we rejected the connection request. Digging deeper, it appears that the underlying SSH2 library allows us to reject a connection request BUT specify that we will accept a request with a different security type. I added the logic to request either password or publickey if a request for none arrives. So instead of a rejection, we move on to subsequent tests. However, even though this fixed problem number 1, problem number 2 surfaced.

We have run into a puzzle that is related to interactions with the underlying SSH2 library used in the project and FileZilla. It is referred to in this issue:

https://github.com/mscdex/ssh2/issues/914

In testing last year, we tested FileZilla with gcs-sftp using a security mechanism of password (and hence bypassed the issue described above). However, it failed then too. This was tracked in the SSH2 issue described above. What we find is that FileZilla expects the requests it sends to be processed by the SFTP server serially in the order they are received. This seems to have resulted in FileZilla expecting the server to process requests serially. The SSH2 library apparently processed requests parallely and the author of that library has documented that the SFTP protocol allows parallel activity and doesn't plan to make any changes to the library. Since all our work is based on the SSH2 library, we appear to be stuck for a resolution.