search

veeso / termscp

🖥 A feature rich terminal UI file transfer and explorer with support for SCP/SFTP/FTP/S3/SMB
https://termscp.veeso.dev
MIT License
1.61k stars 46 forks source link

Decouple file transfers into a separate crate #69

Closed ok-nick closed 2 years ago

ok-nick commented 2 years ago

TermSCP's generic file transfer implementation is extremely well developed and one of a kind! AFAIK, there isn't a single crate dedicated to supporting a generic impl of multiple file transfer protocols over the network. Decoupling this would hopefully establish a general ground in the Rust ecosystem and encourage the community to implement their own protocols.

I'm sure this isn't a top priority, but nevertheless, it is definitely something that should be considered.

ok-nick commented 2 years ago

Elaborating on the idea of an external crate, it would probably be more extensible to split the networking portions from the actual file system into separate traits. I just found rust-vfs, which seems to be similar to what I'm talking about, although it has fewer features and hasn't been maintained in a while.

veeso commented 2 years ago

Hi there!

Actually I've been thinking about this idea for a few months and I'm glad someone else has proposed it 😄 , because it means that it might be worth it.

Clearly it's not a priority right now, but I'll start working on it as soon as I've got some spare time.

Regarding the filesystem stuff I think it makes sense, clearly I think it would make everything a bit more complicated, but surely it's very interesting. I'll give it a look for sure 😉

ok-nick commented 2 years ago

I have a use case for this, so I'd definitely be interested in contributing if you are able to get the repo setup.

veeso commented 2 years ago

Here you go: https://github.com/veeso/remotefs-rs.

A quick guide on how to continue the development:

  1. First FileTransfer has been renamed to RemoteFileSystem and everything refers to Remote something.
  2. I've ported the File module from termscp into this, but I removed the prefix Fs so now FsEntry is just Entry and the same for File and Directory.
  3. I've created the SshKeyStorage trait to work with storage. This just has one method to resolve the host/username and returns a path to the key. We don't have to implement a storage, but it must be the argument passed to Sftp and Scp connectors as dyn something, maybe in a Box, so the final user implements the storage by himself.
  4. At this point it should be enough to implement the drivers. We can just copy them from termscp, but mind that many functions has changed name or scope to be more fs-friendly.
  5. I'm not using vfs since it provides functions for I/O with the Seek trait, but we can't use seek with remote files, so I just re-implemented the trait.
veeso commented 2 years ago

I close the issue since we're working on it.

ok-nick commented 2 years ago

chicon-rs seems to accomplish the same goal as remotefs-rs, although it is more generalized to any file system. It also seems to follow the same conventions as the native std::fs lib which is nice. The only downside is that it is outdated and could use some design changes, but I think its a good reference.

What do you think about it?

veeso commented 2 years ago

I gave a look to chicon, but tbh it has some critical performance issues:

I think that remotefs is already fine and I would keep working on the current trait. Then when I'll have the time to actually work on it, I will redesign some things (naming, paths, ...). Let's consider that 95% is already done, thanks to termscp.

Unfortunately I still can't work on it, since in my spare time, which is a few hours a week, I'm totally on a new api for my project tui-realm and I won't have enough for another month I think.

veeso commented 2 years ago

remotefs has just been released!