search

sickcodes / Docker-OSX

Run macOS VM in a Docker! Run near native OSX-KVM in Docker! X11 Forwarding! CI/CD for OS X Security Research! Docker mac Containers.
https://hub.docker.com/r/sickcodes/docker-osx
GNU General Public License v3.0
48.82k stars 2.73k forks source link

In Memory Panic VNC #96

Open gruaig opened 4 years ago

gruaig commented 4 years ago

Hey base OS Centos 8 Docker version 19.03.5, build 633a0ea QEMU emulator version 1.5.3 (qemu-kvm-1.5.3-173.el7_8.3), Copyright (c) 2003-2008 Fabrice Bellard jenkins-osx ~]# lscpu | grep Virtualization Virtualization type: full lsmod | grep -i kvm kvm 636931 0 irqbypass 13503 1 kvm

docker command ' docker run -e RAM=4 --name docker-osx-vnc -p 8888:5999 -p 50922:10022 -d --privileged docker-osx-vnc '

Screenshot from 2020-10-15 12-23-36 Screenshot from 2020-10-15 12-23-18

While installing over VNC it hits the above error message and freezes let me know if you need any more information

sickcodes commented 4 years ago

Testing now

sickcodes commented 4 years ago

Try rebuilding it with no cache

docker pull sickcodes/docker-osx:latest
docker build --no-cache -t docker-osx-vnc .

I'm running

https://rpmfind.net/linux/RPM/centos/updates/7.8.2003/x86_64/Packages/qemu-kvm-1.5.3-173.el7_8.3.x86_64.html

Package says it has qemu-kvm-5.1.0-7.1.x86_64.rpm which looks good

Were you able to install?

If it didn't boot at all, then hardware nested virt might be off

egrep -c '(svm|vmx)' /proc/cpuinfo
uname -a
sickcodes commented 4 years ago

I can't replicate it as the OS won't download lol, will try again later without VPN

UliPrantz commented 4 years ago

Got the same problem but also only got qemu 2.11.1, which is considered the newest version by apt. Is there any easy way to install qemu v5.0.1 under ubuntu? I only found this guide but building it manually seems a little bit too much

UliPrantz commented 4 years ago

Tried with new qemu version and still got the same error:

Output of virsh -c qemu:///system version --daemon is:

Compiled against library: libvirt 4.0.0
Using library: libvirt 4.0.0
Using API: QEMU 4.0.0
Running hypervisor: QEMU 5.1.0
Running against daemon: 4.0.0
sickcodes commented 4 years ago

@UliPrantz You should install qemu latest from the ubuntu repo. If you don't have qemu 5 in ubuntu, consider upgrading the distro ti latest.

Virsh and qemu are inside the Docker image btw

jamestalbot3 commented 3 years ago

Was running into this problem today. Crapped out at the exact same point. Using the alternate run command solved it for me. Not sure which command did it, but just use trial and error.

emsi commented 3 years ago

Same here though on regular images:

image

Both big sur and catalina images stared as:

docker run -it --device /dev/kvm -p 50922:10022 -v /tmp/.X11-unix:/tmp/.X11-unix -e "DISPLAY=${DISPLAY:-:0.0}" sickcodes/docker-osx:big-sur

My workstation has 2 CPUs with 8 cores (16 threads each). Usually it's not OK to use more than 16 vCPUs for virtualization in my setup. Virtualization is enabled:

egrep -c '(svm|vmx)' /proc/cpuinfo
32
emsi commented 3 years ago

m'kay... seems my virtualization beast despite having 128GB of ram and 32 threads does not sport avx2:

emsi@ubu:~$ egrep -c 'avx' /proc/cpuinfo
32
emsi@ubu:~$ egrep -c 'avx2' /proc/cpuinfo
0

As per https://github.com/kholia/OSX-KVM

A CPU with SSE4.1 support is required for >= macOS Sierra A CPU with AVX2 support is required for >= macOS Mojave

It might be worth to add CPU checks to container startup.

sickcodes commented 3 years ago

Never seen that error before @emsi thank you for pointing that out! I'll check it shortly using digital ocean

fcastilloec commented 3 years ago

I'm seeing the exact same error with both Big Sur and Catalina and my CPU does have AVX2:

$ egrep -c 'avx2' /proc/cpuinfo
4
$ egrep -c '(svm|vmx)' /proc/cpuinfo
8
sickcodes commented 3 years ago

I'm seeing the exact same error with both Big Sur and Catalina and my CPU does have AVX2:


$ egrep -c 'avx2' /proc/cpuinfo

4

$ egrep -c '(svm|vmx)' /proc/cpuinfo

8

Can you post the output of https://github.com/sickcodes/Docker-OSX/blob/master/.github/ISSUE_TEMPLATE/issue-running-docker-osx.md

fcastilloec commented 3 years ago

Can you post the output of https://github.com/sickcodes/Docker-OSX/blob/master/.github/ISSUE_TEMPLATE/issue-running-docker-osx.md

Here it is, in the same order as the template:

Linux felipe-desktop 5.8.0-45-generic #51~20.04.1-Ubuntu SMP Tue Feb 23 13:46:31 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
:1
1
NAME="Ubuntu"
PRETTY_NAME="Ubuntu 20.04.2 LTS"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
Filesystem      Size  Used Avail Use% Mounted on
/dev/nvme0n1p2  228G  114G  103G  53% /
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.14)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers
libvirtd (libvirt) 6.0.0
              total        used        free      shared  buff/cache   available
Mem:           31Gi       6.8Gi       2.3Gi       560Mi        22Gi        23Gi
Swap:         8.0Gi          0B       8.0Gi
4
8
crw-rw----+ 1 root kvm 10, 232 Mar 18 13:22 /dev/kvm
total 16K
drwxrwxrwt  2 root root 4.0K Mar 18 13:24 .
drwxrwxrwt 32 root root  12K Mar 19 10:43 ..
srwxrwxrwx  1 root root    0 Mar 18 13:22 X0
srwxrwxrwx  1 root root    0 Mar 18 13:24 X1
root        1264  0.1  0.3 1316432 122248 ?      Ssl  Mar18   1:35 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

The output of df might be useless since I've set docker with data-root in a different drive, here's the output of the other drive:

Filesystem                    Size  Used Avail Use% Mounted on
/dev/mapper/home_vg-home_ext  963G  556G  359G  61% /mnt/Docker

I've also used this repository ppa:jacob/virtualisation to update QEMU and libvirtd to these values and the issue still happens:

QEMU emulator version 5.0.0 (Debian 1:5.0-5ubuntu6~ppa0)
Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers
libvirtd (libvirt) 6.6.0
sickcodes commented 3 years ago

chmod 666 /dev/kvm

You also did not post grep "docker\|kvm\|virt" /etc/group

fcastilloec commented 3 years ago

Sorry, I guess I didn't select the last line when copy/pasting. Here it is:

$ grep "docker\|kvm\|virt" /etc/group
kvm:x:108:felipe
docker:x:137:felipe
libvirt-dnsmasq:x:140:

Also, I don't think it's necessary to do chmod 666 /dev/kvm because I'm part of the kvm group, so I should be able to read/write. No need to give access to other users.

redfive2012 commented 3 years ago

I've used the Ubuntu Focal Backports PPA repo to update QEMU and libvirt, and I'm also getting this crash when trying to start the Big Sur install process. Ubuntu server 20.04.2 LTS 64-bit, dual 6-core Intel CPUs w/Hyperthread (total 24 logical), 32GB RAM. I can give any further details if requested. AVX is supported, but apparently AVX2 is not.

sickcodes commented 3 years ago

Have you docker pulled the latest image?

redfive2012 commented 3 years ago

Yes, I've only been trying this in the last couple of days.

On Mon, Jun 28, 2021 at 1:38 PM sickcodes @.***> wrote:

Have you docker pulled the latest image?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/sickcodes/Docker-OSX/issues/96#issuecomment-869880497, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKMNTUNU7EPESSOUI3V7Z3TVCXPZANCNFSM4SR4FSCA .

sickcodes commented 3 years ago

@redfive2012 

# OS related issued, please help us identify the issue by posting the output of this
uname -a \
; echo "${DISPLAY}" \
; echo 1 | sudo tee /sys/module/kvm/parameters/ignore_msrs \
; grep NAME /etc/os-release \
; df -h . \
; qemu-system-x86_64 --version \
; libvirtd --version \
; free -mh \
; nproc \
; egrep -c '(svm|vmx)' /proc/cpuinfo \
; ls -lha /dev/kvm \
; ls -lha /tmp/.X11-unix/ \
; ps aux | grep dockerd \
; docker ps | grep osx \
; grep "docker\|kvm\|virt" /etc/group
redfive2012 commented 3 years ago

@.:~$ uname -a Linux dagobah.newrepublic.homeip.net 5.4.0-77-generic #86-Ubuntu SMP Thu Jun 17 02:35:03 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux @.:~$ echo "${DISPLAY}"

@.:~$ echo 1 | sudo tee /sys/module/kvm/parameters/ignore_msrs [sudo] password for nelson: 1 @.:~$ grep NAME /etc/os-release NAME="Ubuntu" PRETTY_NAME="Ubuntu 20.04.2 LTS" VERSION_CODENAME=focal UBUNTU_CODENAME=focal @.:~$ df -h . Filesystem Size Used Avail Use% Mounted on /dev/sda2 110G 22G 83G 22% / @.:~$ qemu-system-x86_64 --version QEMU emulator version 5.2.0 (Debian 1:5.2+dfsg-9ubuntu3~backport20.04-202104240125~ubuntu20.04.1) Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers @.:~$ libvirtd --version libvirtd (libvirt) 7.0.0 @.:~$ free -mh total used free shared buff/cache available Mem: 31Gi 7.9Gi 376Mi 11Mi 23Gi 23Gi Swap: 8.0Gi 16Mi 8.0Gi @.:~$ nproc 24 @.:~$ egrep -c '{svm|vmx}' /proc/cpuinfo 0 @.:~$ ls -lha /dev/kvm crw-rw---- 1 root kvm 10, 232 Jun 26 21:53 /dev/kvm @.:~$ ls -lha /tmp/.X11-unix/ total 8.0K drwxrwxrwt 2 root root 4.0K Jun 26 21:53 . drwxrwxrwt 19 root root 4.0K Jul 5 10:57 .. @.:~$ ps aux | grep dockerd root 1848 0.2 0.3 3460036 113584 ? Ssl Jun26 34:08 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock nelson 1904825 0.0 0.0 5192 2404 pts/0 S+ 10:58 0:00 grep --color=auto dockerd @.:~$ docker ps | grep osx @.:~$ grep "docker|kvm|virt" /etc/group kvm:x:108:nelson docker:x:122:netdata,nelson libvirt:x:125:nelson libvirt-qemu:x:64055:libvirt-qemu,nelson libvirt-dnsmasq:x:126: @.:~$

Nelson Butterworth

On Thu, Jul 1, 2021 at 4:06 AM sickcodes @.***> wrote:

@redfive2012 https://github.com/redfive2012

OS related issued, please help us identify the issue by posting the output of this

uname -a \ ; echo "${DISPLAY}" \ ; echo 1 | sudo tee /sys/module/kvm/parameters/ignore_msrs \ ; grep NAME /etc/os-release \ ; df -h . \ ; qemu-system-x86_64 --version \ ; libvirtd --version \ ; free -mh \ ; nproc \ ; egrep -c '(svm|vmx)' /proc/cpuinfo \ ; ls -lha /dev/kvm \ ; ls -lha /tmp/.X11-unix/ \ ; ps aux | grep dockerd \ ; docker ps | grep osx \ ; grep "docker|kvm|virt" /etc/group

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/sickcodes/Docker-OSX/issues/96#issuecomment-872022168, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKMNTQQJFR7BKXHFA2HXX3TVQOWXANCNFSM4SR4FSCA .

sangdrax8 commented 3 years ago

Ubunt 20.04, I was running big-sir image a month ago. I tried to boot it up in the last week and couldn't get it to boot. I removed all images and tried pulling clean, but resulted in this same crash.

NAME="Ubuntu"
PRETTY_NAME="Ubuntu 20.04.1 LTS (fossa-bulbasaur X54.1)"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
Filesystem                 Size  Used Avail Use% Mounted on
/dev/mapper/vgubuntu-root  467G  224G  219G  51% /
QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.18)
Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers
libvirtd (libvirt) 6.0.0
              total        used        free      shared  buff/cache   available
Mem:           31Gi       4.8Gi       2.3Gi       2.0Gi        23Gi        23Gi
Swap:         979Mi       2.0Mi       977Mi
8
16
crw-rw----+ 1 root kvm 10, 232 Nov  4 07:42 /dev/kvm
total 16K
drwxrwxrwt  2 root   root   4.0K Nov  4 07:42 .
drwxrwxrwt 33 root   root    12K Nov  5 10:34 ..
srwxrwxrwx  1 brians brians    0 Nov  4 07:42 X0
srwxrwxr-x  1 gdm    gdm       0 Nov  4 07:42 X1024
srwxrwxr-x  1 gdm    gdm       0 Nov  4 07:42 X1025
root        3343  0.3  0.2 1901888 84316 ?       Ssl  Nov04   5:31 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
brians    160013  0.0  0.0   9040  2540 pts/0    S+   10:35   0:00 grep --color=auto dockerd
kvm:x:108:brians
libvirt:x:134:brians
libvirt-qemu:x:64055:libvirt-qemu
libvirt-dnsmasq:x:135:
docker:x:998:brians