Script hangs when creating images

[drewsk011]

Hi;

I re-ran the entire script from scratch (had tried 'stages' earlier. I left the command running overnight: 'macos-guest-virtualbox.sh' with not options.

The system I'm using for virtualization is a bit older, but is fine with all other virtuals (I have Sierra installed as a prepackaged system, but also lots of other Unices). There are lots of resources - Core i7 (4/8); 12gb RAM; 1TB mirrored (fast) HDD, etc. No other processes running. After 6 hours:

Splitting the several-GB InstallESDDmg.pkg into 1GB parts because VirtualBox hasn't implemented UDF/HFS VISO support yet and macOS doesn't support ISO 9660 Level 3 with files larger than 2GB. Converting to BaseSystem.dmg to BaseSystem.img Killed root@Drew-PC:/home/drewsk# ls -l total 15293164 -rw-r--r-- 1 root root 11968 Sep 21 2018 ApfsDriverLoader.efi -rw-r--r-- 1 root root 10116 Sep 21 2018 AppleImageLoader.efi -rw-rw-rw- 1 drewsk drewsk 43214 Sep 21 2018 AppleSupport-v2.0.4-RELEASE.zip -rw-r--r-- 1 root root 46656 Sep 21 2018 AppleUiSupport.efi -rw-rw-rw- 1 drewsk drewsk 328 Oct 18 09:01 Catalina_AppleDiagnostics.chunklist -rw-rw-rw- 1 drewsk drewsk 3037852 Oct 18 09:01 Catalina_AppleDiagnostics.dmg -rw-rw-rw- 1 drewsk drewsk 2020 Oct 18 09:01 Catalina_BaseSystem.chunklist -rw-rw-rw- 1 drewsk drewsk 499014051 Oct 18 09:02 Catalina_BaseSystem.dmg -rw-rw-rw- 1 drewsk drewsk 1000000000 Oct 27 01:50 Catalina_InstallESD.part00 -rw-rw-rw- 1 drewsk drewsk 1000000000 Oct 27 01:51 Catalina_InstallESD.part01 -rw-rw-rw- 1 drewsk drewsk 1000000000 Oct 27 01:52 Catalina_InstallESD.part02 -rw-rw-rw- 1 drewsk drewsk 1000000000 Oct 27 01:52 Catalina_InstallESD.part03 -rw-rw-rw- 1 drewsk drewsk 1000000000 Oct 27 01:53 Catalina_InstallESD.part04 -rw-rw-rw- 1 drewsk drewsk 1000000000 Oct 27 01:53 Catalina_InstallESD.part05 -rw-rw-rw- 1 drewsk drewsk 1000000000 Oct 27 01:54 Catalina_InstallESD.part06 -rw-rw-rw- 1 drewsk drewsk 578129164 Oct 27 01:54 Catalina_InstallESD.part07 -rw-rw-rw- 1 drewsk drewsk 7578129164 Oct 18 09:13 Catalina_InstallESDDmg.pkg -rw-rw-rw- 1 drewsk drewsk 1584 Oct 18 09:13 Catalina_InstallInfo.plist -rw-rw-rw- 1 drewsk drewsk 872 Oct 27 01:54 Catalina_installation_files.viso drwxrwxr-x 1 drewsk drewsk 512 Nov 21 2017 dmg2img-1.6.7 -rwxrwxrwx 1 drewsk drewsk 102400 Oct 26 16:50 dmg2img_1.6.7.orig.tar -rwxrwxrwx 1 drewsk drewsk 38775 Oct 27 01:49 macos-guest-virtualbox.sh

I can only assume the "Converting to BaseSystem.dmg to BaseSystem.img" was ouput when I killed the script. All prerequisites are installed. This is WSL, ubuntu.

Any suggestions?

Thanks, Drew.

[myspaghetti]

Thanks for the detailed bug report.

• Try removing all other versions of dmg2img and installing it through the system's package manager (apt install dmg2img)
• Try running dmg2img -v -d Catalina_BaseSystem.dmg BaseSystem.vdi and post the output dmg2img.log file here.

[drewsk011]

Hi; The two copies of dmg2img you saw in the file listing were simply the sources. I'd decided I didn't really want to bootstrap gcc inside a WSL Linux, so they were there. The installed version is dmg2img from apt.

The script ran fine with the options you suggested:

root@Drew-PC:/home/drewsk# dmg2img -v -d Catalina_BaseSystem.dmg BaseSystem.vdi

dmg2img v1.6.7 (c) vu1tur (to@vu1tur.eu.org)

Catalina_BaseSystem.dmg --> BaseSystem.vdi

Debug info will be written to dmg2img.log

reading property list, 45143 bytes from address 498968396 ... partition 0: begin=203, size=430, decoded=284 partition 1: begin=948, size=430, decoded=284 partition 2: begin=1695, size=430, decoded=284 partition 3: begin=2424, size=430, decoded=284 partition 4: begin=3137, size=35530, decoded=23924 partition 5: begin=38950, size=430, decoded=284 partition 6: begin=39678, size=430, decoded=284 partition 7: begin=40423, size=430, decoded=284

decompressing: opening partition 0 ... [2] 100.00% ok opening partition 1 ... [2] 100.00% ok opening partition 2 ... [2] 100.00% ok opening partition 3 ... [2] 100.00% ok opening partition 4 ... [593] 100.00% ok opening partition 5 ... [2] 100.00% ok opening partition 6 ... [2] 100.00% ok opening partition 7 ... [2] 100.00% ok

Archive successfully decompressed as BaseSystem.vdi root@Drew-PC:/home/drewsk#

dmg2img.log

[drewsk011]

Of other note: System is Windows 10 (Insider) - Slow Ring: Current release: 1903 (18362.10024)

[myspaghetti]

It might be a permissions issue since you're running the script as root; try running the script without elevated privileges.

I mistakenly used BaseSystem.vdi when I should have named the file BaseSystem.img. Please execute the following:

mv BaseSystem.vdi Catalina_BaseSystem.img
VBoxManage convertfromraw --format VDI "Catalina_BaseSystem.img" "Catalina_BaseSystem.vdi"

This will allow you to continue the script. You can run it from the beginning or you can run only these stages:

./macos_guest_virtualbox.sh create_target_vdi create_install_vdi attach_initial_storage configure_vm populate_virtual_disks prepare_the_installer_app start_the_installer_app place_efi_apfs_drivers detach_installer_vdi_and_viso boot_macos_and_clean_up

Even if it works at this point, further investigation is required. Unfortunately it'll be a long time before I'll get around to setting up a Windows 10 1903 (18362.10024) test machine.

Thanks for the bug report!

[drewsk011]

Hi;

Perhaps the file system sizes have changed for Catalina? I've managed to clear all of the M$ Windows errors (another, separate port on that with tips for other users will follow in a few hours) then get the second boot of Catalina which hangs here (even if retried). [See drews-catalina1.jpg].

The script on Windows [WSL Ubuntu] halts here:

Shutting down virtual machine. Press enter when the virtual machine shutdown is complete. Waiting for VM "macOS Catalina-10.15" to power on... VM "macOS Catalina-10.15" has been successfully started.

Press enter when the Language window is ready.

Press enter when the macOS Utilities window is ready.

Press enter when the Terminal command prompt is ready.

Installer started. When the installer finishes preparing, the virtual machine will reboot into the base system again, not the installer.

After the VM boots, press enter when either the Language window or Utilities window is ready.

Drew.

[drewsk011]

[myspaghetti]

If you don't mind, please run the following in the virtual machine terminal:

md5 /Install*/Contents/SharedSupport/InstallESD.dmg

And run the following on the host in the script's working directory:

md5sum InstallESDDmg.pkg

If dmg2img hangs when run with the script but succeeds when run manually, and then there are further issues, and then InstallESD.dmg is corrupt, it might be a permissions issue or a read/write issue.

[myspaghetti]

The md5sum of the latest Catalina InstallESD.dmg is 6485ea0551b9f912fd6a1616d19d8ae4

[drewsk011]

Hi;

Attached are two jpg's. One shows the md5 is different, but check the second; assignment for working space - showing 100% usage. I can't see in the script where you set the disk space (I assigned 40gb for the catalina install) - and when I restarted this after cleaning the windows issues, everything was done from scratch.

[drewsk011]

Sorry, assigned 50gb; but there is no breakout for partitions; and there's that new Catalina update that pushed today - could also have been other add ons that I don't know about

[myspaghetti]

InstallESD is supposed to show 100% usage, it's read-only. The base system, /, has free space, and the target, /Volumes/macOS (or in your case /Volumes/macOS Catalina-10.15) is empty until the installer is copied to it.

I re-downloaded the Catalina installation files. These are the checksums for installer version 10.15.1.0.0.1571878017:

md5sum 997813b7d53201b77c8ed74af9b0b80b *Catalina_BaseSystem.dmg
sha1sum 95cc0c151a0e81099c02a8155f19fbf483acc064 *Catalina_BaseSystem.dmg
sha256sum 4f06ee218ab3da5df9f6dc98217f0eb28d2c249c49c747f9a36d1b68455e63e0 *Catalina_BaseSystem.dmg

md5sum 6485ea0551b9f912fd6a1616d19d8ae4 *Catalina_InstallESDDmg.pkg
sha1sum f7179a49c4b775800edfdb8b92aaa6cdfd8b20fc *Catalina_InstallESDDmg.pkg
sha256sum d3976c4e1aa006f27581ead22357275a85b071e66070bc3ccc5a633b76b9f695 *Catalina_InstallESDDmg.pkg

If your checksums differ, you will need to re-download the files.

[drewsk011]

Hi; Thanks - got everything running and Catalina looks great. First and foremost thanks for taking the time to make such a great installer for VirtualBox.

Here's some debug that I tracked down that can squash a couple of bugs:

If you re-run the script and these files exist, the script will hang on part07 (no errors) -rw-rw-rw- 1 drewsk drewsk 1000000000 Oct 30 09:15 Catalina_InstallESD.part00 -rw-rw-rw- 1 drewsk drewsk 1000000000 Oct 30 09:16 Catalina_InstallESD.part01 -rw-rw-rw- 1 drewsk drewsk 1000000000 Oct 30 09:17 Catalina_InstallESD.part02 -rw-rw-rw- 1 drewsk drewsk 1000000000 Oct 30 09:17 Catalina_InstallESD.part03 -rw-rw-rw- 1 drewsk drewsk 1000000000 Oct 30 09:18 Catalina_InstallESD.part04 -rw-rw-rw- 1 drewsk drewsk 1000000000 Oct 30 09:18 Catalina_InstallESD.part05 -rw-rw-rw- 1 drewsk drewsk 1000000000 Oct 30 09:19 Catalina_InstallESD.part06 -rw-rw-rw- 1 drewsk drewsk 616729867 Oct 30 09:19 Catalina_InstallESD.part07

I guess it doesn't close the file handles.

When you reach this point, the terminal window issues a "halt" to MacOS.

Shutting down virtual machine. Press enter when the virtual machine shutdown is complete.

You may want to clarify that the VirtualBox VM will shutdown (but may be a minute or so). If you kill the Virtual Host (Catalina) here, you will endlessly cycle into the MacOS Utilities window and installation won't complete.

On first boot, you should hit enter to execute the startup script.

Do you want me to post some stuff about squashing M$ bugs with virtualbox (as I'm using 1903 and it's fresh for me).

I'm now off to find the last version of OS-X with Rosetta. Wish me luck. Ask away and thanks again for the script.

[myspaghetti]

Closing file handles is far too low-level for a bash script. It seems all the issues reported are either filesystem issues or download issues. I don't plan on handling any low-level filesystem issues or adding download checksums, so I'm closing this as "wontfix"