Sparse volume - Githubissues

johanneswuerbach commented 8 years ago

WIP implementation of sparse volumes as disks

Requires: https://github.com/mist64/xhyve/pull/80

zchee commented 8 years ago

@johanneswuerbach Wow... I will test.

zchee commented 8 years ago

Build success. And I have not yet much to test, but works file. Wonderful!

But if we are connecting an external HDD, rdisk2 in occured error.

johanneswuerbach commented 8 years ago

Yes, the disk number will definitely be parsed from the out of hdiutil attach. Real volumes may not work as I never tested them.

zchee commented 8 years ago

@johanneswuerbach Ok. Do you have any plans to fix this PR? If true, I will wait.

johanneswuerbach commented 8 years ago

Yep, I plan to finish it today.

zchee commented 8 years ago

@johanneswuerbach Thanks :) I'm looking forward to fix.

johanneswuerbach commented 8 years ago

Ready :+1:

zchee commented 8 years ago

@johanneswuerbach Thanks! I will test it.

zchee commented 8 years ago

@johanneswuerbach I tested.

When MACHINE_DRIVER_DEBUG=1 make run command, works fine. but, when creating VM in a standard way, e.g.

docker-machine create --driver xhyve --xhyve-cpu-count 4 --xhyve-memory-size 8196 --xhyve-disk-size 20000 --xhyve-experimental-nfs-share xhyve-test

An error has occurred in the first SSH login.

The difference is only the volume size.

make run: 2000 (2GB)
standard way: 20000 (20GB)

I do not yet debugging, but does this problem occur in your environment?

Result log:

> docker-machine create --driver xhyve --xhyve-cpu-count 4 --xhyve-memory-size 8196 --xhyve-disk-size 20000 --xhyve-experimental-nfs-share xhyve-test
Docker Machine Version:  0.6.0-dev, build 626bd0d
Found binary path at /usr/local/bin/docker-machine-driver-xhyve
Launching plugin server for driver xhyve
Plugin server listening at address 127.0.0.1:57873
() DBG | operation not supported by device
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(flag-lookup) Calling .GetCreateFlags
Found binary path at /usr/local/bin/docker-machine-driver-xhyve
Launching plugin server for driver xhyve
Plugin server listening at address 127.0.0.1:57877
() DBG | operation not supported by device
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(xhyve-test) Calling .GetMachineName
(xhyve-test) Calling .DriverName
(xhyve-test) Calling .GetCreateFlags
(xhyve-test) Calling .SetConfigFromFlags
Running pre-create checks...
(xhyve-test) Calling .PreCreateCheck
(xhyve-test) DBG | ===== Docker Machine xhyve Driver Version 0.1.0 (5e71450) =====
(xhyve-test) DBG |
(xhyve-test) DBG | executing: /usr/local/bin/VBoxManage -v
(xhyve-test) DBG | STDOUT: 5.0.12r104815
(xhyve-test) DBG |
(xhyve-test) DBG | STDERR:
(xhyve-test) Calling .GetConfigRaw
Creating machine...
(xhyve-test) Calling .Create
(xhyve-test) Downloading /Users/zchee/.docker/machine/cache/boot2docker.iso from /Users/zchee/.docker/machine/cache/boot2docker.iso...
(xhyve-test) Creating VM...
(xhyve-test) Extracting vmlinuz64 and initrd.img from boot2docker.iso...
(xhyve-test) DBG | Mounting boot2docker.iso
(xhyve-test) DBG | executing: &{/usr/bin/hdiutil [hdiutil attach /Users/zchee/.docker/machine/machines/xhyve-test/boot2docker.iso -mountpoint /Users/zchee/.docker/machine/machines/xhyve-test/b2d-image] []  <nil> 0xc82008a008 0xc82008a010 [] <nil> <nil> <nil> <nil> false [] [] [] [] <nil>} attach /Users/zchee/.docker/machine/machines/xhyve-test/boot2docker.iso -mountpoint /Users/zchee/.docker/machine/machines/xhyve-test/b2d-image
(xhyve-test) /dev/disk7                                                 /Users/zchee/.docker/machine/machines/xhyve-test/b2d-image
(xhyve-test) DBG | Extracting vmlinuz64 into /Users/zchee/.docker/machine/machines/xhyve-test
(xhyve-test) DBG | Extracting initrd.img into /Users/zchee/.docker/machine/machines/xhyve-test
(xhyve-test) DBG | Unmounting boot2docker.iso
(xhyve-test) DBG | executing: &{/usr/bin/hdiutil [hdiutil detach /Users/zchee/.docker/machine/machines/xhyve-test/b2d-image] []  <nil> 0xc82008a008 0xc82008a010 [] <nil> <nil> <nil> <nil> false [] [] [] [] <nil>} detach /Users/zchee/.docker/machine/machines/xhyve-test/b2d-image
(xhyve-test) "disk7" unmounted.
(xhyve-test) "disk7" ejected.
(xhyve-test) Generating 20000MB disk image...
(xhyve-test) DBG | executing: &{/usr/bin/hdiutil [hdiutil create -megabytes 20000 -type SPARSEBUNDLE /Users/zchee/.docker/machine/machines/xhyve-test/root-volume] []  <nil> 0xc82008a008 0xc82008a010 [] <nil> <nil> <nil> <nil> false [] [] [] [] <nil>} create -megabytes 20000 -type SPARSEBUNDLE /Users/zchee/.docker/machine/machines/xhyve-test/root-volume
(xhyve-test) created: /Users/zchee/.docker/machine/machines/xhyve-test/root-volume.sparsebundle
(xhyve-test) Creating SSH key...
(xhyve-test) Fix file permission...
(xhyve-test) DBG | /Users/zchee/.docker/machine/machines/xhyve-test/boot2docker.iso
(xhyve-test) DBG | /Users/zchee/.docker/machine/machines/xhyve-test/config.json
(xhyve-test) DBG | /Users/zchee/.docker/machine/machines/xhyve-test/id_rsa
(xhyve-test) DBG | /Users/zchee/.docker/machine/machines/xhyve-test/id_rsa.pub
(xhyve-test) DBG | /Users/zchee/.docker/machine/machines/xhyve-test/initrd.img
(xhyve-test) DBG | /Users/zchee/.docker/machine/machines/xhyve-test/root-volume.sparsebundle
(xhyve-test) DBG | /Users/zchee/.docker/machine/machines/xhyve-test/vmlinuz64
(xhyve-test) Generate UUID...
(xhyve-test) DBG | Generated UUID: D6894892-630E-49F7-9097-5B228E162938
(xhyve-test) Convert UUID to MAC address...
(xhyve-test) DBG | Converted MAC address: 12:96:25:11:cf:32
(xhyve-test) Starting xhyve-test...
(xhyve-test) DBG | [xhyve -A -U D6894892-630E-49F7-9097-5B228E162938 -c 4 -m 8196M -l com1,autopty -s 0:0,hostbridge -s 31,lpc -s 2:0,virtio-net -s 3,ahci-cd,/Users/zchee/.docker/machine/machines/xhyve-test/boot2docker.iso -s 4:0,ahci-hd,/dev/rdisk7 -f kexec,/Users/zchee/.docker/machine/machines/xhyve-test/vmlinuz64,/Users/zchee/.docker/machine/machines/xhyve-test/initrd.img,loglevel=3 user=docker console=ttyS0 console=tty0 noembed nomodeset norestore waitusb=10 base host=boot2docker -F /Users/zchee/.docker/machine/machines/xhyve-test/xhyve-test.pid]
(xhyve-test) Waiting for VM to come online...
(xhyve-test) DBG |
(xhyve-test) DBG | Not there yet 1/60, error: IP not found for MAC 12:96:25:11:cf:32 in DHCP leases
(xhyve-test) DBG | operation not supported by device
(xhyve-test) Waiting on a pseudo-terminal to be ready... done
(xhyve-test) Hook up your terminal emulator to /dev/ttys005 in order to connect to your VM
(xhyve-test) DBG |
(xhyve-test) DBG | Not there yet 2/60, error: IP not found for MAC 12:96:25:11:cf:32 in DHCP leases
(xhyve-test) DBG |
(xhyve-test) DBG | Not there yet 3/60, error: IP not found for MAC 12:96:25:11:cf:32 in DHCP leases
(xhyve-test) DBG | rdmsr to register 0x570 on vcpu 0
(xhyve-test) DBG | rdmsr to register 0x570 on vcpu 1
(xhyve-test) DBG | rdmsr to register 0x570 on vcpu 2
(xhyve-test) DBG | rdmsr to register 0x570 on vcpu 3
(xhyve-test) DBG |
(xhyve-test) DBG | Not there yet 4/60, error: IP not found for MAC 12:96:25:11:cf:32 in DHCP leases
(xhyve-test) DBG |
(xhyve-test) DBG | Not there yet 5/60, error: IP not found for MAC 12:96:25:11:cf:32 in DHCP leases
(xhyve-test) DBG |
(xhyve-test) DBG | Not there yet 6/60, error: IP not found for MAC 12:96:25:11:cf:32 in DHCP leases
(xhyve-test) DBG |
(xhyve-test) DBG | Not there yet 7/60, error: IP not found for MAC 12:96:25:11:cf:32 in DHCP leases
(xhyve-test) DBG |
(xhyve-test) DBG | Not there yet 8/60, error: IP not found for MAC 12:96:25:11:cf:32 in DHCP leases
(xhyve-test) DBG |
(xhyve-test) DBG | Not there yet 9/60, error: IP not found for MAC 12:96:25:11:cf:32 in DHCP leases
(xhyve-test) DBG |
(xhyve-test) DBG | Not there yet 10/60, error: IP not found for MAC 12:96:25:11:cf:32 in DHCP leases
(xhyve-test) DBG |
(xhyve-test) DBG | Not there yet 11/60, error: IP not found for MAC 12:96:25:11:cf:32 in DHCP leases
(xhyve-test) DBG | 192.168.64.12
(xhyve-test) DBG | IP found in DHCP lease table: 192.168.64.12
(xhyve-test) DBG | Got an ip: 192.168.64.12
(xhyve-test) DBG | Using SSH client type: external
(xhyve-test) DBG | {[-o BatchMode=yes -o PasswordAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=quiet -o ConnectionAttempts=3 -o ConnectTimeout=10 -o ControlMaster=no -o ControlPath=none docker@192.168.64.12 -o IdentitiesOnly=yes -i /Users/zchee/.docker/machine/machines/xhyve-test/id_rsa -p 22] /usr/bin/ssh}
(xhyve-test) DBG | About to run SSH command:
(xhyve-test) DBG | echo -e "#/bin/bash\nsudo mkdir -p /Users\nsudo /usr/local/etc/init.d/nfs-client start\nsudo mount -t nfs -o noacl,async 192.168.64.1:/Users /Users\n" | sudo tee /var/lib/boot2docker/bootlocal.sh && sudo chmod +x /var/lib/boot2docker/bootlocal.sh && /var/lib/boot2docker/bootlocal.sh
(xhyve-test) DBG | SSH cmd err, output: exit status 255:
(xhyve-test) DBG | NFS setup failed: Something went wrong running an SSH command!
(xhyve-test) DBG | command : echo -e "#/bin/bash\nsudo mkdir -p /Users\nsudo /usr/local/etc/init.d/nfs-client start\nsudo mount -t nfs -o noacl,async 192.168.64.1:/Users /Users\n" | sudo tee /var/lib/boot2docker/bootlocal.sh && sudo chmod +x /var/lib/boot2docker/bootlocal.sh && /var/lib/boot2docker/bootlocal.sh
(xhyve-test) DBG | err     : exit status 255
(xhyve-test) DBG | output  :
(xhyve-test) DBG |
(xhyve-test) Calling .GetConfigRaw
(xhyve-test) Calling .DriverName
Waiting for machine to be running, this may take a few minutes...
(xhyve-test) Calling .GetState
(xhyve-test) Getting to VM state...
Machine is running, waiting for SSH to be available...
Getting to WaitForSSH function...
(xhyve-test) Calling .GetSSHHostname
(xhyve-test) Calling .GetSSHPort
(xhyve-test) Calling .GetSSHKeyPath
(xhyve-test) Calling .GetSSHKeyPath
(xhyve-test) Calling .GetSSHUsername
Using SSH client type: native
{{{<nil> 0 [] [] []} docker [0x51ebe0] <nil>  []} 192.168.64.12 22}
About to run SSH command:
exit 0
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [publickey none], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [publickey none], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [publickey none], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [publickey none], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

zchee commented 8 years ago

@johanneswuerbach When volume size of 25000, sucessful first ssh login. Why... :(

and, When docker pull of some images, no sapce left on device error. Hmm...

pull script:

#!/bin/bash
set -x

docker-machine create --driver xhyve --xhyve-disk-size 25000 xhyve-test
eval "$(docker-machine env xhyve-test)"
docker pull ubuntu:trusty
docker pull ubuntu:wily
docker pull buildpack-deps:trusty
docker pull buildpack-deps:wily

Result:

Docker Machine Version:  0.6.0-dev, build 626bd0d
Found binary path at /usr/local/bin/docker-machine-driver-xhyve
Launching plugin server for driver xhyve
Plugin server listening at address 127.0.0.1:60010
() DBG | operation not supported by device
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(xhyve-test) Calling .GetURL
Reading CA certificate from /Users/zchee/.docker/machine/certs/ca.pem
Reading server certificate from /Users/zchee/.docker/machine/machines/xhyve-test/server.pem
Reading server key from /Users/zchee/.docker/machine/machines/xhyve-test/server-key.pem
Making call to close driver server
(xhyve-test) Calling .Close
Successfully made call to close driver server
Making call to close connection to plugin binary
(xhyve-test) DBG | Closing plugin on server side
trusty: Pulling from library/ubuntu
d89e1bee20d9: Pull complete
9e0bc8a71bde: Pull complete
27aa681c95e5: Pull complete
a3ed95caeb02: Pull complete
Digest: sha256:3a7f4c0573b303f7a36b20ead6190bd81cabf323fc62c77d52fb8fa3e9f7edfe
Status: Downloaded newer image for ubuntu:trusty
wily: Pulling from library/ubuntu
ef748466d532: Pull complete
ce6d2c777319: Pull complete
fd64ed10b137: Pull complete
a3ed95caeb02: Pull complete
Digest: sha256:e9ee37531255adef45b30f63ec36264035f5548c9ae435dfebb2e0076347ad30
Status: Downloaded newer image for ubuntu:wily
trusty: Pulling from library/buildpack-deps
d89e1bee20d9: Already exists
9e0bc8a71bde: Already exists
27aa681c95e5: Already exists
a3ed95caeb02: Already exists
81a56d4235fc: Pull complete
37b228997dc8: Retrying in 9 seconds
44ce77b5b601: Retrying in 9 seconds

# after a short time...

44ce77b5b601: Downloading [==================================================>] 95.61 MB/95.61 MB
write /mnt/sda1/var/lib/docker/0.0/tmp/GetImageBlob886190454: no space left on device
wily: Pulling from library/buildpack-deps
ef748466d532: Already exists
ce6d2c777319: Already exists
fd64ed10b137: Already exists
a3ed95caeb02: Already exists
f71c3e47df8c: Retrying in 1 seconds
64e77a64fa35: Retrying in 1 seconds
a524e9ebec49: Downloading [==================================================>] 133.5 MB/133.5 MB
write /mnt/sda1/var/lib/docker/0.0/tmp/GetImageBlob688918211: no space left on device

johanneswuerbach commented 8 years ago

Sorry, fixed. I used an regular int for storing the number of blocks facepalm. Should work now. I also remove the generated disk image now on Remove() as it is generated by root so docker-machine wasn't able to remove it.

zchee commented 8 years ago

@johanneswuerbach I testing now... But, failed ssh login and no space error is not happening at the moment.

I will continue to test, and I'll try to check whether there is any problem :)

I also remove the generated disk image now on Remove() as it is generated by root so docker-machine wasn't able to remove it.

Yes. I also have recognized. Thanks for fixed.

zchee commented 8 years ago

@johanneswuerbach BTW, I have been waiting for support sparse volume... So, this PR is very very wonderful!!

Thanks again, @johanneswuerbach, awesome job!

zchee commented 8 years ago

@johanneswuerbach I did some of the test, and work perfectly! Merged.

BTW, I think the release of the binary, Do you have any plans feature implementation?

johanneswuerbach commented 8 years ago

No, go ahead :+1:

machine-drivers / docker-machine-driver-xhyve

Sparse volume #44