Running Xcode Build

[maxlapides]

OS related issued, please help us identify the issue by posting the output of this

``` Linux max-ubuntu 5.8.0-63-generic #71~20.04.1-Ubuntu SMP Thu Jul 15 17:46:08 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux 1 NAME="Ubuntu" PRETTY_NAME="Ubuntu 20.04.2 LTS" VERSION_CODENAME=focal UBUNTU_CODENAME=focal Filesystem Size Used Avail Use% Mounted on /dev/nvme0n1p1 938G 499G 392G 56% / QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.17) Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers libvirtd (libvirt) 6.0.0 total used free shared buff/cache available Mem: 15Gi 2.6Gi 159Mi 75Mi 12Gi 12Gi Swap: 2.0Gi 87Mi 1.9Gi 6 12 crw-rw----+ 1 root kvm 10, 232 Jul 23 14:29 /dev/kvm total 8.0K drwxrwxrwt 2 root root 4.0K Jul 22 14:43 . drwxrwxrwt 22 root root 4.0K Jul 23 14:34 .. srwxrwxrwx 1 maxlapides maxlapides 0 Jul 22 14:43 X0 srwxrwxr-x 1 gdm gdm 0 Jul 22 14:43 X1024 srwxrwxr-x 1 gdm gdm 0 Jul 22 14:43 X1025 root 1534 6.4 0.5 1538376 85236 ? Ssl Jul22 92:51 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock maxlapi+ 56608 0.0 0.0 17672 740 pts/1 S+ 14:35 0:00 grep --color=auto dockerd kvm:x:108: docker:x:998:maxlapides ```

Hey there! This is an awesome project. I've successfully created a deduped.img file following the CI/CD instructions.

I am now trying to use this .img file to run my Xcode build. So, here's what I'm running:

docker run -it \
    --device /dev/kvm \
    -p 50922:10022 \
    -e GENERATE_UNIQUE=true \
    -e NOPICKER=true \
    -e MASTER_PLIST_URL=https://raw.githubusercontent.com/sickcodes/Docker-OSX/master/custom/config-nopicker-custom.plist \
    -e IMAGE_PATH=/home/arch/OSX-KVM/mac_hdd_ng.img \
    -e "OSX_COMMANDS=/bin/bash -c \"pwd && uname -a\"" \
    mycontainer

This results in a (qemu) command prompt, but I'm not sure what to do with that. I thought the OSX_COMMANDS would be run and output, but that doesn't seem to happen here. Do you have any recommendations on what my next steps should be?

Thanks again! Really appreciate the work on this!

[maxlapides]

Not sure if this is helpful, but this is what I'm seeing!

[sickcodes]

-e "OSX_COMMANDS=/bin/bash -c \"pwd && uname -a\"" \

Is a demo command, you can change it to something else:

-e "OSX_COMMANDS=/bin/bash -c \"echo hello\""

The container will delete itself after running the commands, or you can remove that line

[maxlapides]

The container will delete itself after running the commands, or you can remove that line

I think this is where I'm a bit confused. Here's what I just ran:

docker run -it \
    --device /dev/kvm \
    -p 50922:10022 \
    -e GENERATE_UNIQUE=true \
    -e NOPICKER=true \
    -e MASTER_PLIST_URL=https://raw.githubusercontent.com/sickcodes/Docker-OSX/master/custom/config-nopicker-custom.plist \
    -e IMAGE_PATH=/home/arch/OSX-KVM/mac_hdd_ng.img \
    -e "OSX_COMMANDS=/bin/bash -c \"echo hello\"" \
    mycontainer

And here's the result:

It never terminates, and I never see "hello" logged. I feel like I'm so so close to having this working!

[sickcodes]

My container will only work in full auto with the OSX_COMMANDS if you make the user account arch and the password alpine.

Try that and you can use the full auto version, but you'll also need to build using the Dockerfile.auto

[maxlapides]

Okay I read through the Dockerfile.auto and I think I understand better now. In the above example, OSX_COMMANDS isn't working because I'm building my Docker image from :naked. Thanks for that tip!

FROM sickcodes/docker-osx:naked
USER arch
COPY --chown=arch ./deduped /image/mac_hdd_ng.img

So, now I'm trying a different approach where I run my docker-osx container by itself and then have my CI tasks SSH into that container to run various scripts. Locally, I am testing out SSH first by running this:

docker run -it \
  --device /dev/kvm \
  -p 50922:10022 \
  -e GENERATE_UNIQUE=true \
  -e NOPICKER=true \
  -e IMAGE_PATH=/image/mac_hdd_ng.img \
  mydockerosximage

And then after that starts up, outside of the container I can run:

sshpass -pmypassword ssh -o StrictHostKeyChecking=no myuser@127.0.0.1 -p 50922 'uname -a'

And that works!

I can also run:

docker exec [MY_CONTAINER_ID] sshpass -pmypassword ssh -o StrictHostKeyChecking=no myuser@127.0.0.1 -p 10022 'uname -a'

This also works 😄

So next I configured my Kubernetes-based CI system to use a readinessProbe to determine whether macOS is ready:

readinessProbe:
  exec:
    command: ["sshpass", "-pmypassword", "ssh", "-o", "StrictHostKeyChecking=no", "myuser@127.0.0.1", "-p", "10022", "'uname -a'"]

I also configured all the same environment variables:

env:
  - name: NOPICKER
    value: 'true'
  - name: GENERATE_UNIQUE
    value: 'true'
  - name: IMAGE_PATH
    value: '/image/mac_hdd_ng.img'

As expected, the probe fails until macOS is ready. It sends this error:

Readiness probe failed: ssh: connect to host 127.0.0.1 port 10022: Connection refused

But ultimately, the probe starts returning a different error:

Readiness probe failed: kex_exchange_identification: read: Connection reset by peer Connection reset by 127.0.0.1 port 10022

And that's where I'm currently stuck! This may be a configuration mistake in my CI system, but I'm not sure right now. Any ideas are welcome :)

[maxlapides]

Oh one more potentially useful clue, the logs for the docker-osx container end like this:

alsa: Could not initialize ADC
alsa: Failed to open `default':
alsa: Reason: No such file or directory
ALSA lib confmisc.c:855:(parse_card) cannot find card '0'
ALSA lib conf.c:5111:(_snd_config_evaluate) function snd_func_card_inum returned error: No such file or directory
ALSA lib confmisc.c:422:(snd_func_concat) error evaluating strings
ALSA lib conf.c:5111:(_snd_config_evaluate) function snd_func_concat returned error: No such file or directory
ALSA lib confmisc.c:1334:(snd_func_refer) error evaluating name
ALSA lib conf.c:5111:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5599:(snd_config_expand) Evaluate error: No such file or directory
ALSA lib pcm.c:2660:(snd_pcm_open_noupdate) Unknown PCM default
alsa: Could not initialize ADC
alsa: Failed to open `default':
alsa: Reason: No such file or directory
audio: Failed to create voice `adc'
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1
qemu-system-x86_64: Slirp: Failed to send packet, ret: -1

[sickcodes]

Remove OSX_COMMANDS to get a shell, only in :auto

Example: https://github.com/sickcodes/Docker-OSX#prebuilt-image-with-arbitrary-command-line-arguments

[Eslam-mohammed-anwar]

@maxlapides Did you end up running it for CICD pipelines , if you please share any heads up

[maxlapides]

@Eslam-mohammed-anwar I did eventually figure out the issue here. The root cause is that the readiness probe that I wrote is no good.