search

ericaltendorf / plotman

Chia plotting manager
Apache License 2.0
908 stars 280 forks source link

Enable local archiving (and improve archiving design) #6

Open ericaltendorf opened 3 years ago

ericaltendorf commented 3 years ago

Currently, the archiving process assumes archiving to a remote host using rsync.

We should allow multiple transports. At least, rysnc between local dirs.

The main complication here is that we not only need to transfer files, but also to check free disk space on the target directories. Currently this is implemented by remotely executing the df command on the remote host. So we need a clean and robust way to check free space that is coordinated with the file transport mechanism.

There's an additional complication because when transferring via rsync to a remote rsync daemon, we see virtual paths as exported by the rsync daemon configured modules, but when we execute df, we view paths as they exist natively on the remote host. We currently have a fairly crufty mechanism for mapping between the two views of paths. This mechanism needs to be robust, and to generalize across whatever transport mechanisms we implement.

mileino commented 3 years ago

As a request, it would be nice to be able to enable rsync over SSH in the config. This would allow for secure transfer over the internet(, of course this would bring encryption overhead). This would allow not needing to have rsync daemon running, which would inturn add security. I would also maybe opt for plotman being configured to use rsync over SSH by default, so that no one by mistake has rsync facing the internet.

Handri-Kosada commented 3 years ago

Will be great if can put the ssh custom port in the config.yaml

I have a case where my plotter machine is outside and need custom ssh port to be secure

ptsmonteiro commented 3 years ago

As a request, it would be nice to be able to enable rsync over SSH in the config. This would allow for secure transfer over the internet(, of course this would bring encryption overhead). This would allow not needing to have rsync daemon running, which would inturn add security. I would also maybe opt for plotman being configured to use rsync over SSH by default, so that no one by mistake has rsync facing the internet.

+1

BasilHorowt commented 3 years ago

This should at least partly resolved by this very active PR : https://github.com/ericaltendorf/plotman/pull/88