search

devopsgroup-io / vagrant-hostmanager

:pencil: A Vagrant plugin that manages hosts files within a multi-machine environment.
Mozilla Public License 2.0
1.46k stars 146 forks source link

hostsfile download truncated by first 16kB #235

Open double-p opened 7 years ago

double-p commented 7 years ago

vagrant/1.9.4 with vagrant-hostmanager/1.8.6 (same effect both on debian9 and OSX10.10) guest-OS: CentOS6.6

I noticed a corruption of /etc/hosts in about 2 of 3 times hostmanager being invoked. The hosts file is around 30kB before and way smaller if hostmanager "fails".

After some knee-deep debugging, I found out that at this moment (hosts_file/updater.rb): machine.communicate.download(realhostfile, file) the downloaded file is already truncated by exactly 16kByte.

Likely there is some race condition (38 VMs in this Vagrantfile) or so.. to qualify this, I halted all but one VM and changed updater.rb as follows:

          file = @global_env.tmp_path.join("hosts.#{machine.name}")
          filebackup = @global_env.tmp_path.join("hosts.backup.#{machine.name}")
          machine.communicate.download(realhostfile, filebackup)
          machine.communicate.download(realhostfile, file)
          @logger.debug("size of first download is: #{filebackup.size}")
          @logger.debug("size of second download is: #{file.size}")

Then I ran a manual loop of "vagrant hostmanager" with a "sudo cp ~/30khosts /etc/hosts" on the guest between the hostmanager calls.

Twelve runs of that result in these filesizes:

 $ VAGRANT_LOG=debug vagrant hostmanager 2>&1|grep 'size of'
DEBUG updater: size of first download is: 15422
DEBUG updater: size of second download is: 15422
 $ VAGRANT_LOG=debug vagrant hostmanager 2>&1|grep 'size of'
DEBUG updater: size of first download is: 15422
DEBUG updater: size of second download is: 15422
 $ VAGRANT_LOG=debug vagrant hostmanager 2>&1|grep 'size of'
DEBUG updater: size of first download is: 15422
DEBUG updater: size of second download is: 31806
 $ VAGRANT_LOG=debug vagrant hostmanager 2>&1|grep 'size of'
DEBUG updater: size of first download is: 15422
DEBUG updater: size of second download is: 31806
 $ VAGRANT_LOG=debug vagrant hostmanager 2>&1|grep 'size of'
DEBUG updater: size of first download is: 15422
DEBUG updater: size of second download is: 15422
 $ VAGRANT_LOG=debug vagrant hostmanager 2>&1|grep 'size of'
DEBUG updater: size of first download is: 15422
DEBUG updater: size of second download is: 15422
 $ VAGRANT_LOG=debug vagrant hostmanager 2>&1|grep 'size of'
DEBUG updater: size of first download is: 15422
DEBUG updater: size of second download is: 31806
 $ VAGRANT_LOG=debug vagrant hostmanager 2>&1|grep 'size of'
DEBUG updater: size of first download is: 15422
DEBUG updater: size of second download is: 15422
 $ VAGRANT_LOG=debug vagrant hostmanager 2>&1|grep 'size of'
DEBUG updater: size of first download is: 15422
DEBUG updater: size of second download is: 15422
 $ VAGRANT_LOG=debug vagrant hostmanager 2>&1|grep 'size of'
DEBUG updater: size of first download is: 15422
DEBUG updater: size of second download is: 15422
 $ VAGRANT_LOG=debug vagrant hostmanager 2>&1|grep 'size of'
DEBUG updater: size of first download is: 15422
DEBUG updater: size of second download is: 31806
 $ VAGRANT_LOG=debug vagrant hostmanager 2>&1|grep 'size of'
DEBUG updater: size of first download is: 15422
DEBUG updater: size of second download is: 15422

As one can see, the first download always being truncated, whereas the second consecutive one is correct sometimes. The difference from 31802 to 15422 is 16384.

Besides the manual 'cp' there are no processes having open fd's on /etc/hosts (lsof check). I checked a DEBUG sshd-logging for (unlikely) connections losses or other errors to no avail. Also tried to disable the Re-using of ssh connection (to add verbose: true), but failed so far.

Any bells ringing/insights/thoughts appreciated.