Low-latency rsync-auto take #2

[dkarlovi]

I've created this issue before with #6399, it got closed with #6568 but I don't feel as it's been fixed and cannot seem to reopen it. I understand that you guys are seeing 100ms syncs in small tests, but I can't replicate that performance.

Please consider the following solution:

1. starting up the Rsync server once, at rsync-auto startup
2. keeping a SSH connection open for as long as rsync-auto is running, I don't think control paths solution did the trick, at least for me, I'm still seeing 2-3sec delays, might be wrong though
3. incorporate the "changed only" strategy done in vagrant-sync-changed-only done by @nuncanada, this is pretty important as people are seing >1min sync times, as seen in #6821
4. terminate the Rsync server and SSH connection only when rsync-auto is stopped

This would fix stuff like

• removing the mental hurdle for web developers like "Why isn't this changing? Is it cached again? What is happ... oh right, rsync. Oh, there we go." every time they refresh the page
• removing the problem with stacked updates because, the way I read it, it will not run syncs in parallel and, if you make two consecutive changes, you'll wait quite a bit to see the second one
• removing the huge sync times in large folders

It should be an all-around win and make rsync THE preferred sync method. Thanks.

[dkarlovi]

Here's a timing test:

• ~30k files
• DHCP networking
• Linux host, Linux guest (VirtualBox)

Running rsync-auto --debug, changing a random file in project root:

INFO rsync-auto: Time spent in rsync: 2.68853676 (in seconds)
INFO rsync-auto: Time spent in rsync: 2.526970299 (in seconds)

Running rsync-auto-only-changed --debug, same type of change:

INFO rsync-auto-only-changed: Time spent in rsync: 1.262107906 (in seconds)
INFO rsync-auto-only-changed: Time spent in rsync: 1.322466242 (in seconds)

[nuncanada]

There is a problem with the current implementation of rsync-auto-only-changed when there is a big quantity of changed/added files (typical when moving a directory, git pull, etc...)

It needs to setup "Event coalescing" as was done in this other plugin: https://github.com/smerrill/vagrant-gatling-rsync

[deancsmith]

We're also seeing a lot more latency with rsync since Vagrant 1.8.x. We previously had rsync times down to 2-3 seconds with some timeout tweaks on our guest setup (from 6-8 seconds) on Vagrant 1.7.x. We're now back to 6-8 seconds on Vagrant 1.8.x although nothing has changed on the guest end (the size of the project does not seem to vary this time by more than second or so).

Our setup is an OS X host and Windows 2008r2 guests with rsync installed using cygwin.

[snfnwgi]

Who can answer my question https://github.com/hashicorp/vagrant/issues/9010

[dswitzer]

I would love to see this implemented. While I'm not seeing huge delays, every save does take 5-15 seconds to propagate, which can feel like forever when you're trying to tweak changes and just want to see the changes quickly take affect.