search

coderanger / kitchen-sync

Test Kitchen transport plugin to speed up tests.
Apache License 2.0
81 stars 24 forks source link

Add sshtar transport #10

Closed i11 closed 9 years ago

i11 commented 9 years ago

TK >= 1.4 transport using ssh and tar piping to improve file transfer performance

coderanger commented 9 years ago

Why would this be faster than SFTP? It should already be multiplexed over a single SSH connection.

coderanger commented 9 years ago

Oh I see, you are using openssh locally instead of Net::SSH. That's flat out worse than using the Rsync method already present as that will do hash checks and compression too and has the same problem of needing a passwordless connection.

i11 commented 9 years ago

Let me disagree that it's "...flat out worse...". It's very minimalistic and doesn't require rsync presents on both local and remote end points. It's definitely similar if not better than rsync in terms of cons. Not to mention that rsync code hasn't been ported to TK 1.4. Possible use-cases with TK are more or less endless and if something doesn't fit your particular approach, doesn't mean it won't benefit others. Suggested transport addresses transmission issues only. File upload time for me went down from 16 minutes to 5 seconds just like that. It becomes irrelevant that files will be re-transmitted, because by the time sftp or rsync are done with checksum calculations my files will be already there.

Anyway I can see the lack of understanding of the approach and need. Thankfully it's not a problem to create a new gem.

Cheers!

coderanger commented 9 years ago

rsync is included by default on most linux distros. The only reason I have the SFTP option at all is that rsync requires a password-less SSH connection like your code does. This is unfortunately a tricky requirement and means it will have to always take a back seat to the in-band systems like SFTP and SCP. I would be very surprised if this performs better than rsync, especially over multiple runs which is the main target for this gem. If you would like to port the rsync code up to 1.4 I would be happy to merge that, but this gem is in limbo until @fnichol gets some time to make the transport plugin API actually usable.

patcon commented 8 years ago

rsync is included by default on most linux distros

fwiw, not centos, apparently