search

linka-cloud / d2vm

Build Virtual Machine Image from Dockerfile or Docker image
Apache License 2.0
208 stars 31 forks source link

MacOS: The file /dev/loop0p1 does not exist and no size was specified. #13

Closed buahaha closed 1 year ago

buahaha commented 1 year ago

Running this command: % sudo ~/go/bin/d2vm build -p 0 -f Dockerfile -o nutek-linux.vdi .

Leads to this error:

Building docker image from /in/Dockerfile
Inspecting image d2vm-cdc6670a-be2d-4db5-9d50-7d1414cad66b
No network manager specified, using distribution defaults: ifupdown
Docker image based on Kali GNU/Linux 2022.3
Building kernel enabled image
Creating vm image
Creating raw image
Mounting raw image
Creating raw image file system
Build failed
Unmounting raw image
Failed to unmount
Error: mkfs.ext4 /dev/loop0p1: stdout:  stderr: mke2fs 1.45.5 (07-Jan-2020)
The file /dev/loop0p1 does not exist and no size was specified.
 error: exit status 1

There is no mkfs.ext4 on MacOS

% which mkfs.ext4
mkfs.ext4 not found
Adphi commented 1 year ago

As mentioned in the README:

Starting from v0.1.0, d2vm automatically run build and convert commands inside Docker when not running on linux or when running without root privileges.

So d2vm uses the container which has mkfs.ext4. The problem here is: The file /dev/loop0p1 does not exist and no size was specified.

Can you re-run your command with the --verbose flag ?

buahaha commented 1 year ago

--verbose output:

$ docker image build -t os-release-d2vm-05661514-5d10-41d0-bd25-7b2c052f435b -f /tmp/d2vm/6c65562d-ea50-4650-801a-90751d7a4ca5/osrelease.Dockerfile /tmp/d2vm/6c65562d-ea50-4650-801a-90751d7a4ca5
#1 [internal] load build definition from osrelease.Dockerfile
#1 sha256:b869b9e8676292fe444f5fc8af3b8faa449db55ece51ebb50a4345941a034abb
#1 transferring dockerfile: 149B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:ed74d7de821969fced1108963aeb7c2bcae20ec2450c6b03e2f2117f9e76a7ea
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/d2vm-05661514-5d10-41d0-bd25-7b2c052f435b:latest
#3 sha256:9d1bd4de278296931546355e5cb1ab1ad17e2988f3cdf7797f564e58f613bd38
#3 DONE 0.0s

#4 [1/1] FROM docker.io/library/d2vm-05661514-5d10-41d0-bd25-7b2c052f435b
#4 sha256:053a1c1bead6cd89427b6ebd44bb2bbdefc0dc9237af9e28be6ffe35c606214a
#4 CACHED

#5 exporting to image
#5 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#5 exporting layers done
#5 writing image sha256:c397ce49dc684ebaaf677884bf414f2895f896bf6dc63ceb71efcbd15592ab0d done
#5 naming to docker.io/library/os-release-d2vm-05661514-5d10-41d0-bd25-7b2c052f435b done
#5 DONE 0.0s
$ docker image rm os-release-d2vm-05661514-5d10-41d0-bd25-7b2c052f435b
Untagged: os-release-d2vm-05661514-5d10-41d0-bd25-7b2c052f435b:latest
Deleted: sha256:c397ce49dc684ebaaf677884bf414f2895f896bf6dc63ceb71efcbd15592ab0d
No network manager specified, using distribution defaults: ifupdown
Docker image based on Kali GNU/Linux 2022.3
Building kernel enabled image
$ docker image build -t 6c65562d-ea50-4650-801a-90751d7a4ca5 -f /tmp/d2vm/6c65562d-ea50-4650-801a-90751d7a4ca5/d2vm-05661514-5d10-41d0-bd25-7b2c052f435b /tmp/d2vm/6c65562d-ea50-4650-801a-90751d7a4ca5
#1 [internal] load build definition from d2vm-05661514-5d10-41d0-bd25-7b2c052f435b
#1 sha256:ed98ace63682cd802f0a5faf0ab2d6cc3efed0bf6f6732f4cc450477a990d155
#1 transferring dockerfile: 761B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:75463f9dc7957e77b79f0639a1e4891ad1cb57d50494bc34181cf581efc7ed23
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/d2vm-05661514-5d10-41d0-bd25-7b2c052f435b:latest
#3 sha256:9d1bd4de278296931546355e5cb1ab1ad17e2988f3cdf7797f564e58f613bd38
#3 DONE 0.0s

#4 [1/7] FROM docker.io/library/d2vm-05661514-5d10-41d0-bd25-7b2c052f435b
#4 sha256:053a1c1bead6cd89427b6ebd44bb2bbdefc0dc9237af9e28be6ffe35c606214a
#4 DONE 0.0s

#6 [3/7] RUN DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends       systemd-sysv       systemd       dbus       iproute2       isc-dhcp-client       iputils-ping
#6 sha256:02418c26b452a958a95908c2a783fb526810db18eaae6b5723f46dac1f841af5
#6 CACHED

#7 [4/7] RUN systemctl preset-all
#7 sha256:c7d7ff3cd3f8888830a49ad7c1feefb7b5a9efec8c241dd74cbcc6626e0c8454
#7 CACHED

#9 [6/7] RUN if [ -z "$(apt-cache madison ifupdown2 2> /dev/nul)" ]; then apt install -y ifupdown; else apt install -y ifupdown2; fi
#9 sha256:460685a0a9bd73f367f01ec858da763cd09a3868a005a265d6aea8cdff546b8a
#9 CACHED

#5 [2/7] RUN apt-get -y update &&     DEBIAN_FRONTEND=noninteractive apt-get -y install --no-install-recommends       linux-image-amd64
#5 sha256:2a292b2b04076d5ce5ce24d2357b6bf3eb340ecb2cf435e732ec1775a2da2f3c
#5 CACHED

#8 [5/7] RUN echo "root:0" | chpasswd
#8 sha256:76a8e2bbb40019f9c655dbac8f84003f03db1eeca3684d455ac4734bfb967aed
#8 CACHED

#10 [7/7] RUN mkdir -p /etc/network && printf 'auto eth0\nallow-hotplug eth0\niface eth0 inet dhcp\n' > /etc/network/interfaces
#10 sha256:c5d018bfd27fe13c83ed3bd9dee4b412b07a4336a70141485ac7962960030061
#10 CACHED

#11 exporting to image
#11 sha256:e8c613e07b0b7ff33893b694f7759a10d42e180f2b4dc349fb57dc6b71dcab00
#11 exporting layers done
#11 writing image sha256:4a0e1cf0e8a4fb0a2b62988773d070aa68a6b47e01a45fc14578d6872d268851 done
#11 naming to docker.io/library/6c65562d-ea50-4650-801a-90751d7a4ca5 done
#11 DONE 0.0s
Creating vm image
Creating raw image
$ parted -s /tmp/d2vm/6c65562d-ea50-4650-801a-90751d7a4ca5/disk0.d2vm.raw mklabel msdos mkpart primary 1Mib 100% set 1 boot on
Mounting raw image
$ partprobe /dev/loop0
Creating raw image file system
$ mkfs.ext4 /dev/loop0p1
mke2fs 1.45.5 (07-Jan-2020)
The file /dev/loop0p1 does not exist and no size was specified.
Build failed
Unmounting raw image
$ umount /tmp/d2vm/6c65562d-ea50-4650-801a-90751d7a4ca5/mnt
umount: /tmp/d2vm/6c65562d-ea50-4650-801a-90751d7a4ca5/mnt: not mounted.
$ losetup -d /dev/loop0
Failed to unmount
$ docker image rm 6c65562d-ea50-4650-801a-90751d7a4ca5
Untagged: 6c65562d-ea50-4650-801a-90751d7a4ca5:latest
Deleted: sha256:4a0e1cf0e8a4fb0a2b62988773d070aa68a6b47e01a45fc14578d6872d268851
Error: exit status 1
Adphi commented 1 year ago

ok, it doesn't give much more information... can you try some of the steps manually inside the docker container and share the output ?

docker run --rm -i -t --privileged --entrypoint bash linkacloud/d2vm:v0.1.0-rc3
# inside the container

# create a sparse volume of 5G
$ fallocate -l 5G /tmp/disk
# create the partition with labels
$ parted -s /tmp/disk mklabel msdos mkpart primary 1Mib 100% set 1 boot on
# display the volume layout
$ fdisk -l /tmp/disk
# list used loop devices
$ losetup
# map the volume to the first available loopback device
$ LOOPDEV=$(losetup --show -f /tmp/disk)
# show which loopback device it has been mapped to
$ echo $LOOPDEV
# scan volume partitions
$ partprobe $LOOPDEV
# list loop devices
$ ls /dev/loop*
# create ext4 filesystem on the first partition
$ mkfs.ext4 ${LOOPDEV}p1
# finally cleanup with (even if the previous command failed)
$ losetup -d $LOOPDEV
buahaha commented 1 year ago

Output of your commands:

 % docker run --rm -i -t --privileged --entrypoint bash linkacloud/d2vm:v0.1.0-rc3
Unable to find image 'linkacloud/d2vm:v0.1.0-rc3' locally
v0.1.0-rc3: Pulling from linkacloud/d2vm
d7bfe07ed847: Already exists 
804fa0f1c513: Pull complete 
e2e1e4f27485: Pull complete 
40bf76dcf7cb: Pull complete 
Digest: sha256:a8717df5df4ece57d9cc90efdc716868e833272fdd2b64cf43e6e63bdf4d9619
Status: Downloaded newer image for linkacloud/d2vm:v0.1.0-rc3
root@6652692f2548:/# fallocate -l 5G /tmp/disk
root@6652692f2548:/# parted -s /tmp/disk mklabel msdos mkpart primary 1Mib 100% set 1 boot on
root@6652692f2548:/# fdisk -l /tmp/disk
Disk /tmp/disk: 5 GiB, 5368709120 bytes, 10485760 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xd4be6929

Device     Boot Start      End  Sectors Size Id Type
/tmp/disk1 *     2048 10485759 10483712   5G 83 Linux
root@6652692f2548:/# losetup
root@6652692f2548:/# LOOPDEV=$(losetup --show -f /tmp/disk)
root@6652692f2548:/# echo $LOOPDEV
/dev/loop0
root@6652692f2548:/# partprobe $LOOPDEV
root@6652692f2548:/# ls /dev/loop*
/dev/loop-control  /dev/loop0p1  /dev/loop2  /dev/loop4  /dev/loop6
/dev/loop0         /dev/loop1    /dev/loop3  /dev/loop5  /dev/loop7
root@6652692f2548:/# mkfs.ext4 ${LOOPDEV}p1
mke2fs 1.45.5 (07-Jan-2020)
Creating filesystem with 1310464 4k blocks and 327680 inodes
Filesystem UUID: e97d5d6e-c01f-4ecc-8a44-af6740eed07d
Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376, 294912, 819200, 884736

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done 

root@6652692f2548:/# losetup -d $LOOPDEV
root@6652692f2548:/#
Adphi commented 1 year ago

ok, everything is working as expected... How did you build / install the d2vm binary ? I guess you used make install or make build-dev as it is in your $GOBIN ?

buahaha commented 1 year ago

make install

buahaha commented 1 year ago

version d2vm version v0.1.0-rc4-dev

Adphi commented 1 year ago

can you try to build the image directly in docker (the tag contains your pull request for kali linux support) ?

docker run --rm -i -t --privileged -v $(pwd):/in -w /in linkacloud/d2vm:v0.1.0-rc4-c665951 build -p 0 -f Dockerfile -o nutek-linux.vdi .
buahaha commented 1 year ago

I can build raw image

Adphi commented 1 year ago

But the error message has nothing to do with the format ... Can you try again ?

buahaha commented 1 year ago

Right now, I get another error

Unable to find image 'linkacloud/d2vm:v0.1.0-rc4-c665951' locally
v0.1.0-rc4-c665951: Pulling from linkacloud/d2vm
d7bfe07ed847: Already exists 
8418bc7876f2: Pull complete 
e880db553261: Pull complete 
0fd61ea6685c: Pull complete 
Digest: sha256:7d3461a01ad49816b9fb82db9eb21d3aa6fb5b7cfcec2279be1d364cc003d634
Status: Downloaded newer image for linkacloud/d2vm:v0.1.0-rc4-c665951
Building docker image from Dockerfile
Error: docker image build -t d2vm-ede38a23-fcd7-470f-85e9-bb9e09802423 -f Dockerfile .: stdout:  stderr: unable to prepare context: unable to evaluate symlinks in Dockerfile path: lstat /in/Dockerfile: no such file or directory
 error: exit status 1
Adphi commented 1 year ago

You need to run that from the directory you first tried to build the nutek-linux.vdi image, where your Dockerfile is.

buahaha commented 1 year ago

I'm actually there

buahaha commented 1 year ago

% ls Dockerfile

Adphi commented 1 year ago

Ah. Can you share your Dockerfile?

buahaha commented 1 year ago

.rw-r--r-- me staff 56 B Sun Sep 18 21:22:36 2022 Dockerfile

buahaha commented 1 year ago

FROM kalilinux/kali-rolling:latest ENTRYPOINT /bin/bash

Adphi commented 1 year ago

Sorry, I forgot to mount the docker socket in the docker command...

docker run --rm -i -t --privileged -v /var/run/docker.sock:/var/run/docker.sock -v $(pwd):/in -w /in linkacloud/d2vm:v0.1.0-rc4-c665951 build -p 0 -f Dockerfile -o nutek-linux.vdi .
buahaha commented 1 year ago

And error log continues:

Building docker image from Dockerfile
Error: docker image build -t d2vm-93b5ac53-5c9d-4f92-9141-57c215e4e5bb -f Dockerfile .: stdout:  stderr: #1 [internal] load build definition from Dockerfile
#1 sha256:534a82ea910847b120f6c4bfced842015fb44cb25580d86a45b955910cf6f84a
#1 transferring dockerfile: 2B 0.0s done
#1 DONE 0.0s
failed to solve with frontend dockerfile.v0: failed to read dockerfile: open /var/lib/docker/tmp/buildkit-mount1713062835/Dockerfile: no such file or directory
 error: exit status 1
buahaha commented 1 year ago

maybe this is the reason dockerfile vs Dockerfile?

Adphi commented 1 year ago

I don't know what's wrong. I cannot reproduce the problem:

mkdir -p ~/tmp/kali && cd ~/tmp/kali
cat <<EOF > Dockerfile
FROM kalilinux/kali-rolling:latest
ENTRYPOINT /bin/bash
EOF
docker run --rm -i -t --privileged -v /var/run/docker.sock:/var/run/docker.sock -v $(pwd):/in -w /in linkacloud/d2vm:v0.1.0-rc4-c665951 build -p 0 -f Dockerfile -o nutek-linux.vdi .
Building docker image from Dockerfile
Inspecting image d2vm-9622f62a-553d-4f1e-8f46-c8f714f2ba25
No network manager specified, using distribution defaults: ifupdown
Docker image based on Kali GNU/Linux 2022.3
Building kernel enabled image
Creating vm image
Creating raw image
Mounting raw image
Creating raw image file system
Copying rootfs to raw image
Setting up rootfs
Installing linux kernel
Unmounting raw image
Writing MBR
Converting to vdi

If the Dockerfile is named Dockerfile you can omit the -f Dockerfile

Adphi commented 1 year ago

It also works on my side with:

mkdir -p ~/tmp/kali && cd ~/tmp/kali
cat <<EOF > Dockerfile
FROM kalilinux/kali-rolling:latest
ENTRYPOINT /bin/bash
EOF
d2vm build -p 0 -o nutek-linux.vdi .

And that works too:

d2vm convert -p 0 -o nutek-linux.vdi kalilinux/kali-rolling:latest --force
buahaha commented 1 year ago

This command worked for me ~/go/bin/d2vm build -p 0 -o nutek-linux.vdi . without sudo and without -f Dockerfile

buahaha commented 1 year ago

Thank you, great app :)

Adphi commented 1 year ago

Thanks. You're welcome.