prepare-iso.sh and El Cap Beta 8

[luciusbono]

Running the prepare-iso.sh script on the latest El Capitan Beta completes without errors, but the OS installer image is invalid. VMWare Fusion just boots to the "OMG I DON'T KNOW WHAT TO DO WHERE IS THE OS?" screen and trying to mount the DMG manually and install it in OSX nets the following error message:

I dug around in the script and in the .DMG but I'm not experienced enough with OSX to know what the trouble is / everything looked pretty normal when I compared it to the 10.10.x .DMGs that I've cooked with this process. Any help appreciated!

[timsutton]

I've had the same issue, although I'm not surprised you get a weird error trying to run the installer from within another booted OS (it's really meant to be only a booted installer).

In the past I've been able to look at the "convert installer app" scripts used by Fusion and Parallels for hints on preparing an installer. So far I haven't narrowed down what's different between DP1 and the DP7 installer. The fact that it doesn't even boot suggests to me that there could actually be some different files between the two.

DP7 also introduced SIP-related tightening down of what's "allowed" as far as booting to other volumes goes. I should really hope that there isn't some EFI-related nvram "secure storage" logic that is making this harder, but it could be tangentially related.

So, more digging definitely necessary.

[ahrenstein]

@timsutton Have you had any luck with figuring out the El Capitan issue? I'm hoping to use your scripts to create an El Capitan box ASAP to begin testing software/code on it.

Thanks

[AntonyBlakey]

Even using createinstallmedia from the 10.11 installer doesn't create a bootable image.

[luciusbono]

Does AutoDMG work with El Cap? I don't have 10.11 installed on my machine yet so I'm unable to verify.

Not ignoring this problem, just in a little over my head in terms of how to help. I suspect I will learn a lot because of this little speed-bump!

[timsutton]

AutoDMG definitely works with 10.11. So one alternative workflow would be to use chilcote/vfuse to convert an AutoDMG-built image into a VM, then use the vmware-vmx builder instead.

This would still require some additional packaging work to mimic the osx-vm-templates results exactly. Joe Chilcote may have some other repos/code that produce the requisite packages, though.

I still haven't had time to look into this deeply enough to understand why the approach we've been using doesn't seem to boot the image anymore. However it's possible that we may need to instead try an approach like what VMware does in their included conversion script - which does something similar, but in quite a different way.

[luciusbono]

I'll have to have one of my co-workers with Fusion 8 to give me a copy of that script so I can stare at it and stare at it and stare at it. Where does the script live in the Fusion App?

[timsutton]

You'll find it at /Applications/VMware\ Fusion.app/Contents/Library/Create\ Mavericks\ Installer.tool - make sure to check the one that comes with 8.0.1, though.

This script was broken in 8.0.0 (where ironically, the fix was to steal the one from the Tech Preview that preceded 8.0.0).

[luciusbono]

Cool! I was able to pull that script down from a co-worker. Was about to post it here and then I realized VMWare probably would not like that (apparently I've been dealing with too many open source tools lately).

[timsutton]

Probably not. The script also wouldn't exactly be a drop-in replacement for prepare_iso.sh - there are some approaches it takes that are quite different from those in my script, but I'm not sure whether such changes are really required.

[luciusbono]

This is a good exercise for me to actually understand what your script is doing, and then actually understand what the VMWare script is doing and then actually understand what OSX needs to be bootstrap'd - I've been using your project in production at my workplace but treating prepare_iso.sh as a grey box. Time to stop being lazy, and who knows, maybe even contribute a little.

[upekkha]

Looks like someone on macrumors was able to find a way to make a bootable ISO.

[timsutton]

Thanks for linking this. I'd be interested to know what specifically about this process makes it work where prepare_iso.sh fails.

A couple ideas: 1) using asr restore instead of "directly" booting the dmg, or 2) the specific CDR master formatting of the ISO (again, instead of a vanilla HFS+ DMG, which VMware has always supported for the officially-supported Darwin guests and their EFI implementation).

Would love to test this, not sure I'll find time as soon as this week, though. If anyone wants to do more digging, that would be welcome!

[AntonyBlakey]

On 14 Oct 2015, at 11:10, Timothy Sutton notifications@github.com wrote:

Thanks for linking this. I'd be interested to know what specifically about this process makes it work where prepare_iso.sh fails.

A couple ideas: 1) using asr restore instead of "directly" booting the dmg, or 2) the specific CDR master formatting of the ISO (again, instead of a vanilla HFS+ DMG, which VMware has always supported for the officially-supported Darwin guests and their EFI implementation).

Would love to test this, not sure I'll find time as soon as this week, though. If anyone wants to do more digging, that would be welcome!

I've rebuilt prepare_iso.sh on the basis of that post. I have it working for 10.8 ... 10.11. Not tested 10.7. The issue isn't the CDR because I don't do that. In the interest of quick response, I've attached my version because it's incorporated into a bigger repo that I've not yet published.

Antony Blakey

Ph: +61 438 840 787

Success is not the key to happiness. Happiness is the key to success. -- Albert Schweitzer

[ahrenstein]

@AntonyBlakey It looks like your attachment got lost in the issue comments. Any chance you can post a link to it as a gist? I'd love to test it out.

Thanks

[luciusbono]

Seconded @AntonyBlakey - would looooove to check out your script.

[AntonyBlakey]

https://gist.github.com/AntonyBlakey/e0116fe82e596c5005d8

On 15 Oct 2015, at 08:03, Lucius Bono notifications@github.com wrote:

Seconded @AntonyBlakey - would looooove to check out your script.

Antony Blakey

Ph: +61 438 840 787

All that is required for evil to triumph is that good men do nothing.

[timsutton]

Thank you! I look forward to checking this out tomorrow. Perhaps others will comment in the meantime.

[cybertk]

@AntonyBlakey's script worked for me to boot up a 10.11 image with VirtualBox.

[timsutton]

Works for me too. Since Antony's script has a ton of small changes, I'd still like to identify what exactly is different about the process that was posted on the macrumors thread that makes it work. I'll start testing small changes, starting with using asr restore instead of the DMG copy, although that seems like an odd requirement to make this work.

[cybertk]

I'm glad to send a PR with the minimal changes, if you do not have much time on this.

[timsutton]

Sure, if you can even verify what those changes actually are, that would be great!

[upekkha]

It really seems to boil down to using asr to restore the BaseSystem. With these changes I get a bootable ElCapitan image.

[AntonyBlakey]

On 16 Oct 2015, at 22:29, Claude Becker notifications@github.com wrote:

It really seems to boil down to using asr to restore the BaseSystem. With these changes I get a bootable ElCapitan image.

This line:

umount /Volumes/OS\ X\ Base\ System/

won't work for 10.9 and prior, because they mount using a different name, as you can see in my script.

Note also that my script takes care to cleanup using traps - you might want to add that facility to your version as it's generally useful. With crashes in the script I found stuff chewing up lots of space in /tmp

Antony Blakey

Ph: +61 438 840 787

He who would make his own liberty secure, must guard even his enemy from repression. -- Thomas Paine

[upekkha]

I didn’t submit a PullRequest on purpose. This was only meant to illustrate a possible, smaller variant of changes to allow easier investigation.

On 16 Oct 2015, at 14:18, Antony Blakey notifications@github.com wrote:

On 16 Oct 2015, at 22:29, Claude Becker notifications@github.com wrote:

It really seems to boil down to using asr to restore the BaseSystem. With these changes I get a bootable ElCapitan image.

This line:

umount /Volumes/OS\ X\ Base\ System/

won't work for 10.9 and prior, because they mount using a different name, as you can see in my script.

Note also that my script takes care to cleanup using traps - you might want to add that facility to your version as it's generally useful. With crashes in the script I found stuff chewing up lots of space in /tmp

Antony Blakey

Ph: +61 438 840 787

He who would make his own liberty secure, must guard even his enemy from repression. -- Thomas Paine

— Reply to this email directly or view it on GitHub https://github.com/timsutton/osx-vm-templates/issues/40#issuecomment-148701866.

[timsutton]

@upekkha Thanks for confirming the asr bit. I had a suspicion that was it, though it's curious that enough changed between DP1 and DP7 that using other standard disk image conversion methods no longer works. Maybe someday we'll find out why.

@AntonyBlakey Totally with you about using traps; I didn't start learning about those until long after I first wrote this script, and they're extremely useful for any script that involves mounting and converting DMGs.

[timsutton]

Thank you @AntonyBlakey and @upekkha for your examples. Working fine here. Question for @AntonyBlakey:

May I incorporate a similar set of changes into the prepare_iso.sh script? I'd at least like to adopt the asr restore and the check for the differing volume names from <=10.9 and >=10.10, so a handful of lines. I'll otherwise keep the same structure in the script, and look at improving the cleanup later.

[AntonyBlakey]

On 17 Oct 2015, at 02:57, Timothy Sutton notifications@github.com wrote:

Thank you @AntonyBlakey and @upekkha for your examples. Working fine here. Question for @AntonyBlakey:

May I incorporate a similar set of changes into the prepare_iso.sh script? I'd at least like to adopt the asr restore and the check for the differing volume names from <=10.9 and >=10.10, so a handful of lines. I'll otherwise keep the same structure in the script, and look at improving the cleanup later.

Take whatever you want. I'm going to publish on github with packer setup for all the OS X and Windows versions that I've done for my employer (which is every non-EOL version of both OS's).

Antony Blakey

Ph: +61 438 840 787

The greatest challenge to any thinker is stating the problem in a way that will allow a solution -- Bertrand Russell

[AntonyBlakey]

I've spent a lot of time automating everything. It was a lot of work. Everything will be PD, so you can take what you want. I'll be publishing tomorrow, Australian time.

On Saturday, 17 October 2015, Matthew Ahrenstein notifications@github.com wrote:

@AntonyBlakey https://github.com/AntonyBlakey I'm looking forward to seeing that repo. I'm about to start working on Windows templates as well, so it'd be nice to have a good example to work from.

— Reply to this email directly or view it on GitHub https://github.com/timsutton/osx-vm-templates/issues/40#issuecomment-148775258 .

[ahrenstein]

@AntonyBlakey Awesome thanks!

Not sure where my comment went :/

[timsutton]

@AntonyBlakey: One other question: you also mention the cdrom sometimes being the first disk device here:

https://gist.github.com/AntonyBlakey/e0116fe82e596c5005d8#file-prepare_iso-sh-L233

Does this happen due to a particular configuration in your Packer/automation project? I haven't seen this happen before - though I can imagine it happening if there is some additional work you do in your template with the attached disk devices.

[luciusbono]

This has been soooooo enlightening thank you @AntonyBlakey. I've integrated your script into my production test environment and we'll be able to deploy El Cap VMs next week after I finalize some other tweaks. Comparing these two scripts has really helped me understand this process better, and I'm REALLY interested in your Windows deployments. On Fri, Oct 16, 2015 at 11:52 AM Timothy Sutton notifications@github.com wrote:

@AntonyBlakey https://github.com/AntonyBlakey: One other question: you also mention the cdrom sometimes being the first disk device here:

https://gist.github.com/AntonyBlakey/e0116fe82e596c5005d8#file-prepare_iso-sh-L233

Does this happen due to a particular configuration in your Packer/automation project? I haven't seen this happen before - though I can imagine it happening if there is some additional work you do in your template with the attached disk devices.

— Reply to this email directly or view it on GitHub https://github.com/timsutton/osx-vm-templates/issues/40#issuecomment-148805631 .

[AntonyBlakey]

No idea - I got that bit from your script. I didn't clean up that section much.

I'll add appropriate credit link BTW when I publish.

On Saturday, 17 October 2015, Timothy Sutton notifications@github.com wrote:

@AntonyBlakey https://github.com/AntonyBlakey: One other question: you also mention the cdrom sometimes being the first disk device here:

https://gist.github.com/AntonyBlakey/e0116fe82e596c5005d8#file-prepare_iso-sh-L233

Does this happen due to a particular configuration in your Packer/automation project? I haven't seen this happen before - though I can imagine it happening if there is some additional work you do in your template with the attached disk devices.

— Reply to this email directly or view it on GitHub https://github.com/timsutton/osx-vm-templates/issues/40#issuecomment-148805631 .

[timsutton]

All merged. Thanks!

@AntonyBlakey: Right, but my script makes the assumption that the HD is the first device. I was curious what cases caused you to have it as the second rather than the first. Maybe I'll see once your repo is published, though :)

[AntonyBlakey]

For those interested: https://github.com/AntonyBlakey/packer-platforms

On 18 Oct 2015, at 00:39, Timothy Sutton notifications@github.com wrote:

All merged. Thanks!

@AntonyBlakey: Right, but my script makes the assumption that the HD is the first device. I was curious what cases caused you to have it as the second rather than the first. Maybe I'll see once your repo is published, though :)

— Reply to this email directly or view it on GitHub.

Antony Blakey

Ph: +61 438 840 787

A Man may make a Remark – In itself – a quiet thing That may furnish the Fuse unto a Spark In dormant nature – lain –

Let us divide – with skill – Let us discourse – with care – Powder exists in Charcoal – Before it exists in Fire –

-– Emily Dickinson 913 (1865)