super softwareupdate at login window with no GUI user logged in consistently hangs at black screen with Apple logo and progress bar

[bdresher]

We are experiencing a regular, repeatable issue with S.U.P.E.R.M.A.N. 4.0.3 and we're hoping to find a solution.

And firstly I want to say that super is such a great tool, we are really enjoying using it. I wish Apple would get software update back to where it was years ago, but thanks Kevin for super!

Background

When our super workflow runs if there is no user logged in at the GUI of the computer, as expected there is no prompt, the computer is silently updated with any available updates. The super logs show super runs, downloads the update, prepares the update, and then issues a restart. The computer then goes to a black screen with the Apple logo and progress bar, but just sits at that point and doesn't restart. It appears the Mac will sit in this state forever, although I haven't let it run forever; a few days seemed long enough.

Manual Intervention

We can SSH in and issue a restart command or can often issue a restart with ARD. After the restart, the update appears to be applied and eventually the Mac returns to the login window updated to macOS 14.5. This is a little concerning issuing a reboot at this screen as I never know if it's updating the OS, or firmware, or something else that will lead to bricking the computer. A manual reboot at this point has not bricked a machine for us yet.

Current environment

Apple Silicon hardware, MacBook Airs/Pros [M2, M2 Pro, M3 Pro, M3 Max] & Mac minis [M1, M2, M2 Pro]. We are experiencing this with macOS 14.4.1 which is attempting to update to macOS 14.5. We have been experiencing it for several macOS releases, but it is becoming critical for our environment to have this working properly. We can reproduce the issue by DFU'ing the Mac back to 14.4.1 and then running it through our super workflow again once it's minimally provisioned and enrolled in our JAMF system. We are using a local roleaccount which is setup to be our super account and is a volume owner to authenticate a software update on Apple Silicon. All that stuff appears to work, it's just the restart that never happens.

What we have tried

We've tried running super in verbose mode to get more details in the logs, and have tried adjusting some of the timeout variables in the super script prior to running the workflow. This hasn't seemed to make any difference. The logs do show super re-issuing its workflow every 2 minutes trying to reissue the install with a restart, but the restart never happens.

What does work

Super and software updates behave fine if a user is logged in at the GUI and the user triggers an update through the super dialog or after the deferrals have expired. We have not seen it get stuck in the same manner. Furthermore, we found that if we minimally provision a machine in the same manner but instead of running super we SSH in and elevate to a root shell with sudo /bin/bash and then issue a softwareupdate -i -a --restart, provide the secure token enabled user password, the update downloads, and prepares, stdout appears much the same way super logs record, but this method successfully restarts, without hanging at the Apple logo screen.

Attachments

super-logs.zip

[Macjutsu]

First, thank you for a detailed issue post!

So here the responsible line in super: echo "${auth_local_password}" | sudo -u root softwareupdate --install "${macos_msu_label}" --restart --force --no-scan --agree-to-license --user "${auth_local_account}" --stdinpass >>"${MSU_WORKFLOW_LOG}" 2>&1 &

This line in super differs in two fundamental ways than what works in your testing:

1. Per man softwareupdate the --stdinpass option allows for passing in the password. The fact that the update and restart works at all implies that the softwareupdate command was satisfied with the password.
2. In your tests even though you were not logged in to the GUI... by using SSH as the admin... you were in fact logged in to the shell. However, if super runs on its own it's very possible that no one is logged in to the GUI or shell.

At this point I would say that that 2 is the culprit. It's possible that Apple doesn't support local update/upgrades with no one logged in at all... I doubt they even QA that. A true test would be to start a background script that first waits for 60 seconds (or however long it takes for you to log out after starting the script) and then runs a command similar to super.

At this point more testing is required... but it's possible without some seriously risky scripting hacks (like literally temporarily starting a shell for the admin just to issue the softwareupdate command) we might be approaching something Apple no longer supports.

[bdresher]

Kevin,

I believe we have found the spot in the code that is causing the issue reported in this thread. It appears that the super code has an error on super 4.0.3 line 2947. The formatting of the date -j -f is invalid for the input it is receiving, and as a result the epoch time is not being output. This improper formatting of the epoch time is leading to a logic error causing the comparison to be invalid in the following line of code [[ -n "I${mac_last_startup_saved_epoch}" ]] && [[ -n "\${mac_last_startup_epoch}" ]] && [[ '|${mac_last startup saved epoch}" -ge "\${mac_last_startup_epoch}" ]] && exit 0

Here is an example:

Line in error ("%b %d %H:%M:%S" is what appears to be incorrect, this formatting is invalid in regards to outputting the epoch time): mac_last_startup_epoch=$(date -j -f "%b %d %H:%M:%S" "$(last reboot | head -1 | cut -c 41- | xargs):00" +"%s" 2> /dev/null)

Sample run of the command by hand of the existing code with the improper format string is "%b %d %H:%M:%S": % date -j -f "%b %d %H:%M:%S" "Mon May 05 09:14:18" +"%s" Failed conversion of ``Mon May 05 09:14:18'' using format ``%b %d %H:%M:%S'' date: illegal time format usage: date [-jnRu] [-I[date|hours|minutes|seconds]] [-f input_fmt] [-r filename|seconds] [-v[+|-]val[y|m|w|d|H|M|S]] [[[[mm]dd]HH]MM[[cc]yy][.SS] | new_date] [+output_fmt]

Sample run of the command by hand of the modified code with the proper format string as "%a %B %d %H:%M:%S": % date -j -f "%a %B %d %H:%M:%S" "Mon May 05 09:14:18" +"%s" 1714922058

In context, here are screenshots of the code changes we made that appear to have fixed this issue, before and after:

4.0.3 code (line 2947)

Modified 4.0.3 code (line 2947):

We have modified the script we are using to incorporate these changes. We can confirm that the change fixes the issue described with a software update hanging while the computer is at the login window. We would love to see this change incorporated into the next build of super.

[Macjutsu]

Again, thank you for your detailed response!

That being said... in super v4.1.0-beta1 I replaced the uptime method in favor of a sysctl method. Apple literally broke uptime in macOS 14.4 (see the macOS 14.5 updates for enterprise).

Please check out v4.1.0-beta1.

[bdresher]

I can report v4.1.0-beta1 does resolve the issue we reported with super updating software at the login window. This is so very appreciated to have this working!

[Macjutsu]

Thanks... that's great news!