startosinstall Arg Suggestions & Notes

[PicoMitchell]

I am working on a script to be used with macOS refurbishment, and your startosinstall argument was an invaluable reference. I wanted to share back the related work that I did.

Instead of using OS version checks in my code to determine supported startosinstall arguments, I wanted to use the output of startosinstall --usage. Sadly, I found that startosinstall --usage can take a VERY long time so I went about finding another similar solution. I landed on grepping the binary contents to check for supported arguments. This feels a bit unconventional but its effective and MUCH faster than loading the --usage.

Checking the Darwin major version, as erase-install is doing, is a perfectly valid solution but I thought this alternate solution may interest you.

Also, while I was working on this, I noticed that you have CORRECT code but an INCORRECT comment at https://github.com/grahampugh/erase-install/blob/0e5c38bdb052d270fd8f3c07040adcea408d1d86/erase-install.sh#L1101 This comment is incorrect, although the code correctly only adds the --allowremoval argument on macOS 10.15 Catalina. You can see my comments in the included code for more info on this argument (which also reference your blog post about it).

And lastly, I see that your code in erase-install only sets --stdinpass and --user when running on Apple Silicon. This does not sync with my findings during my own testing. I found that Big Sur requires --stdinpass or --passprompt when on Intel Macs as well. I believe an error is returned stating that specifying form of authentication is required. And if not running as an admin user the --user argument is required to specify which admin to authenticate with (or else it will try to authenticate with the currently running non-admin and fail). I can double-check these findings though since most of my testing was on a non-admin account as well as a T2 Mac which may have had its own requirements separate from non-T2 Intel Macs.

Here is my startosinstall code with the intelligent argument selection based around grepping the startosinstall binary:

startosinstall_args=( '--nointeraction' ) # The "nointeraction" argument is undocumented, but is supported on ALL versions of "startosinstall" (which is OS X 10.11 El Capitan and newer).

# NOTE: Instead of using version checks to determine supported "startosinstall" arguments, I wanted to use the output of "startosinstall --usage" instead.
# But, I found that "startosinstall --usage" can take a VERY long time. Grepping the binary contents to check for supported arguments is unconventional but effective and MUCH faster!
# The part of the binary contents that have the actual usage strings all end with a ", " so it is included in the greps to not match other error messages, etc.
# To retrieve/verify all the strings referencing arguments within a "startosinstall" binary, run: strings "${path_to_installer_app}/Contents/Resources/startosinstall" | grep -e '--'

if grep -qU -e '--agreetolicense, ' "${startosinstall_path}"; then
    # The "agreetolicense" argument is supported on macOS 10.12 Sierra and newer.
    # Although, installations of OS X 10.11 El Capitan and older should still be silent because of the "nointeraction" argument.
    startosinstall_args+=( '--agreetolicense' )
fi

if grep -qU -e '--applicationpath, ' "${startosinstall_path}"; then
    # The "applicationpath" argument is only supported on macOS 10.13 High Sierra and older.
    # I believe "applicationpath" is actually only required for macOS 10.12 Sierra and older, but macOS 10.13 High Sierra still supports it.
    startosinstall_args+=( '--applicationpath' "${startosinstall_path%.app/*}.app" )
fi

if grep -qU -e '--installpackage, ' "${startosinstall_path}"; then
    # The "installpackage" argument is supported on macOS 10.13 High Sierra and newer.
    did_add_installpackage=false
    if [[ -n "${installpackage_paths}" ]]; then
        IFS=$'\n'
        for this_installpackage_path in $installpackage_paths; do
            if [[ -n "${this_installpackage_path}" && -f "${this_installpackage_path}" && "${this_installpackage_path}" == *'.pkg' ]]; then
                startosinstall_args+=( '--installpackage' "${this_installpackage_path}" )
                did_add_installpackage=true
            fi
        done
        unset IFS
    fi

    if ! $did_add_installpackage; then
        echo -e '    WARNING: PERFORMING *CLEAN INSTALL* SINCE NO PACKAGES PROVIDED\n'
    fi
else
    echo -e '    WARNING: PERFORMING *CLEAN INSTALL* SINCE "installpackage" IS NOT SUPPORTED\n'
fi

if grep -qU -e '--forcequitapps, ' "${startosinstall_path}"; then
    # The "forcequitapps" argument is supported on macOS 10.15 Catalina and newer.
    # This should not be necessary IN RECOVERY OS, but doesn't hurt and is useful IN FULL OS.
    startosinstall_args+=( '--forcequitapps' )
fi

if [[ -n "${caffeinate_pid}" ]] && grep -qU -e '--pidtosignal, ' "${startosinstall_path}"; then
    # The "pidtosignal" argument (to terminate the specified PID when the prepare phase is complete) is supported on macOS 10.12 Sierra and newer.
    startosinstall_args+=( '--pidtosignal' "${caffeinate_pid}" )
fi

can_run_startosinstall=false

if $IS_RECOVERY_OS && grep -qU -e '--volume, ' "${startosinstall_path}"; then # Double check this even though it should be redundant if we got this far.
    # The "volume" argument is supported IN RECOVERY OS on ALL versions of "startosinstall" (which is OS X 10.11 El Capitan and newer).
    # The "volume" argument can also be used IN FULL OS when SIP is disabled, but that is not useful or necessary for our usage.
    startosinstall_args+=( '--volume' "${install_volume_path}" )
    can_run_startosinstall=true
elif ! $IS_RECOVERY_OS && grep -qU -e '--eraseinstall, ' "${startosinstall_path}"; then
    # The "eraseinstall" AND "newvolumename" arguments are supported IN FULL OS on macOS 10.13 High Sierra and newer. Also, "eraseinstall" is only supported when booted into an APFS volume.
    # If attemped to run on a non-APFS volume, macOS 10.14 Mojave and newer return "Error: Erase installs are supported only on APFS disks." and macOS 10.13 High Sierra returns "Error: 801".
    startosinstall_args+=( '--eraseinstall' )

    if grep -qU -e '--newvolumename, ' "${startosinstall_path}"; then
        startosinstall_args+=( '--newvolumename' "${install_volume_name}" )
    fi

    if grep -qU 'add --allowremoval.' "${startosinstall_path}"; then
        # The undocumented "allowremoval" argument is only supported for macOS 10.15 Catalina and there is a note in the macOS 11 Big Sur installer that it is ignored and should be removed.
        # NOTICE: Since this argument is undocumented, it is grepped for at the end of a sentence rather than from the usage like all other greps.
        # This specified string will not exist in the macOS 11 Big Sur installer, so it properly won't get added on macOS 11 Big Sur.
        # More Info: https://grahamrpugh.com/2020/06/09/startosinstall-undocumented-options.html#allowremoval
        startosinstall_args+=( '--allowremoval' )
    fi

    if grep -qU -e '--passprompt, ' "${startosinstall_path}"; then
        # The "passprompt" argument (to specify the form of authentication) is required on macOS 11 Big Sur when using the "agreetolicense" argument. Previous versions would just do a GUI prompt when needed.
        # This argument will do a command line prompt on macOS 11 Big Sur. The other authentication option is "stdinpass" which is not useful or necessary for our usage.
        # If "agreetolicense" is used without "passprompt" or "stdinpass" then "Error: A method of password entry is required." will be returned.
        startosinstall_args+=( '--passprompt' )

        if grep -qU -e '--user, ' "${startosinstall_path}"; then
            # The "user" argument must also be included along with "passprompt" (if not running as admin).
            # If not running as admin and no "user" argument is specified, then will always fail with "Error: could not get authorization..." since the current user cannot authorize.
            # Including the username even if running as admin doesn't hurt and needs to be specified if not fully logged in and running via "su" (but without "sudo").
            admin_username="$(id -un)" # We will use the current username as authorizing admin user if it is and admin and has a Secure Token.
            all_admin_usernames="$(dscl . -read /Groups/admin GroupMembership | cut -c 18- | xargs)"
            if [[ " ${all_admin_usernames} " != *" ${admin_username} "* || "$(sysadminctl -secureTokenStatus "${admin_username}" 2>&1)" != *'is ENABLED for'* ]]; then
                IFS=' '
                for this_admin_username in $all_admin_usernames; do
                    if [[ "${this_admin_username}" != 'root' ]]; then
                        if [[ "$(sysadminctl -secureTokenStatus "${this_admin_username}" 2>&1)" == *'is ENABLED for'* ]]; then
                            # If current user was not admin or did not have a Secure Token,
                            # check all admin users for Secure Tokens and use the first admin with a Secure Token.
                            admin_username="${this_admin_username}"
                            break
                        elif [[ " ${all_admin_usernames} " != *" ${admin_username} "* ]]; then
                            # If no admins have a Secure Token because not booted to APFS volume, the "eraseinstall" will exit with an error
                            # because it can only be done on APFS volumes, but still want to use the correct admin username regardless.
                            # So in this case we will fallback to using the first admin or the current user if they are admin.
                            admin_username="${this_admin_username}"
                        fi
                    fi
                done
                unset IFS
            fi

            startosinstall_args+=( '--user' "${admin_username}" ) 

            # Let the technician know which admin password is required since the installer prompt does not specify it.
            echo -e "    NOTICE: You will be prompted for ${admin_username}'s administrator password...\n"
        fi
    fi

    can_run_startosinstall=true
fi

if $can_run_startosinstall; then
    echo -e "    DEBUG startosinstall_args: ${startosinstall_args[*]}\n"
    read -rp '    DEBUG - PRESS ENTER TO RUN startosinstall: '

    "${startosinstall_path}" "${startosinstall_args[@]}"
else
    # Should never get here, but handle it with an error and show usage just in case.

    missing_required_argument='volume'
    if ! $IS_RECOVERY_OS && grep -qU -e '--eraseinstall, ' "${startosinstall_path}"; then missing_required_argument='eraseinstall'; fi

    echo -e "\n  ERROR: THIS \"startosinstall\" DOES NOT SUPPORT THE REQUIRED \"${missing_required_argument}\" ARGUMENT\n"
    "${startosinstall_path}" --usage
fi

[PicoMitchell]

I can double-check these findings though since most of my testing was on a non-admin account as well as a T2 Mac which may have had its own requirements separate from non-T2 Intel Macs.

Big Sur startosinstall behavior confirmed on both T2 and non-T2 Intel Macs:

Big Sur startosinstall with NO arguments will prompt for password via CLI. If NOT running as admin, this always fails with Error: could not get authorization... regardless of entering the current non-admin user password or an admin password. When running as admin, the current admin password can be entered and the installation will start.

When ONLY --agreetolicense is specified, Big Sur startosinstall returns Error: A method of password entry is required.

Using BOTH --agreetolicense AND --passprompt (or --stdinpass) solves that error, but has the same authentication behavior and same non-admin problem as running with no arguments.

The best all around solution is to always include --passprompt or --stdinpass WITH a known good admin user with a Secure Token in the --user argument. If the currently running user is known to be an admin with a Secure Token, the --user argument could be left out, but including it does not hurt anything.

PS. I have updated my code above with more specific comments based on these latest findings in the --passprompt and --user sections.

[grahampugh]

I haven't seen your issue on Intel Macs with Big Sur, but I'll test it in case that's come in on more recent versions of Big Sur.

I'm curious as to your preference for using passprompt. I couldn't get this to work at all, but maybe I did not understand the required syntax. stdinpass works fine for me though, so I went with that.

I'm not sure I see the worth of all the code associated with checking the required arguments of startosinstall, because the erase-install script needs to check for version anyway to determine the validity of the installer. So the check is already done. It seems to take a bunch more code to do it your way. Still, it's an interesting project.

[grahampugh]

I have just run sudo erase-install.sh --reinstall on a T1 Mac running macOS 11.2.3 and it worked. I did not supply the --user or --stdinpass arguments and was not asked for any credentials. I'd be curious to see your output. Are you not using sudo?

[PicoMitchell]

Odd that you have not see the same "authorization" errors on Intel Macs that I was getting in Big Sur. Did you do testing in a non-admin account? I did all my testing in 11.2, so maybe something recently changed. All my testing on that was consistent and reproducible.

For our uses in refurbishment, this tool would generally be run in Recovery OS and would not need the eraseinstall option. But, in the case that someone did already set up a Mac manually for some reason with an arbitrary user account, or our Tech Support department wanted to use this tool to do a re-install on a customer computer, it could launched in the full OS by a technician. In this case, the admin username and password may not be known in advance and the technician would be in front of the computer when the installation starts. In this scenario, using passprompt so that the password prompt comes up within Terminal when needed was the most straight forward approach.

For erase-install, which is more intended to be run unattended, passprompt does not seems like it would be useful since it would wait for user input.

As I said originally, I think checking the version and applying the known correct arguments is a fine way to go. Using grep in this way mostly came about because, other than your script, it was hard to find a definitive source for which arguments applied to which versions of startosinstall. So, I kind of built my investigation into the code. I also though this way would be good for some future-proofing. If thing change, this code makes no assumptions about "this version or newer", it is all about what is actually available in the startosinstall binary that is being used. Of course that doesn't replace actual testing and updating when new versions of macOS come out, but it may help notice subtle changes that could be easy to miss otherwise.

It's fair that each other these conditions take a bit more code than simply checking the version. If this resulted in a noticeable slowness at runtime, I would not have gone with this approach. I also find it nice that each condition makes it absolutely clear which argument each block of code is for. In term of brevity and clarity, I was a bit curious why you repeated bash string manipulations such as ${installer_build:0:2} over and over rather than creating a new more obvious argument such as installer_darwin_major_version="${installer_build:0:2}". Probably doesn't make a noticeable difference at runtime though.

All in all, I just thought it may be interesting for you to see another approach to a similar problem.

[PicoMitchell]

I have just run sudo erase-install.sh --reinstall on a T1 Mac running macOS 11.2.3 and it worked. I did not supply the --user or --stdinpass arguments and was not asked for any credentials. I'd be curious to see your output. Are you not using sudo?

No, I am not running startosinstall as sudo since that would not be useful when running within a non-admin account. That is probably the difference. If erase-install will always be run as root or with sudo, then maybe it is a non-issue for this project.

[grahampugh]

For sure it was interesting, so thanks for sharing!

I've been testing most versions of Big Sur since they came out, so I would be surprised if 11.2 specifically had asked for the user and passprompt and I missed it. I've had that code in the script since testing on our DTK probably before the main Big Sur release.

I suppose I normally test with an admin account rather than a standard account. I think the only standard account testing I have done is on Apple Silicon Macs. But I've had no reports that startosinstall requires user/pass on Intel other than yours. Indeed, many people are still using the EraseInstall application (not related to my script) successfully on Intel Big Sur Macs, and that does not include the prompts.

[grahampugh]

No, I am not running startosinstall as sudo since that would not be useful when running within a non-admin account. That is probably the difference. If erase-install will always be run as root or with sudo, then maybe it is a non-issue for this project.

Aha. Yes, erase-install.sh is envisaged for the main part as a management script that you deploy with your management software, e.g. Jamf Pro or Munki. It is not designed to be run as a standard user.