hang on macOS system upgrade

reini-1 commented 1 year ago

Erroneous Behavior

When there is a macOS system upgrade, topgrade downloads the update but did not continue to install it.

── 10:42:48 - macOS App Store ──────────────────────────────────────────────────
Warning: Nothing found to upgrade

── 10:42:49 - macOS system update ──────────────────────────────────────────────
Software Update Tool

Finding available software
Downloading macOS Ventura 13.5.2
Password:

Downloaded: macOS Ventura 13.5.2

After this nothing happens and topgrade did not continue.

I have to manually open "System Settings" → "General" → "Software Update" and do the update manually.

After the reboot I can call topgrade again and it finishes because there is no other system update.

Expected Behavior

The update gets installed and the computer is restarted.

Steps to reproduce

Wait until there is a new macOS system update and run topgrade on a mac silicon (m1).

Possible Cause (Optional)

Problem persists without calling from topgrade

[ ] Yes
[x] No

Did you run topgrade through `Remote Execution`

[ ] Yes
[x] No

If yes, does the issue still occur when you run topgrade directlly in your remote host

[ ] Yes
[ ] No

Configuration file (Optional)

% rg -v '^$|^#' .config/topgrade.toml
1:[misc]
3:assume_yes = true
7:disable = ["myrepos", "containers", "ruby_gems"]
11:ignore_failures = ["vim"]
17:no_retry = true
23:remote_topgrades = ["primary.multipass", "octoprint.lan", "octoprint4.lan"]
26:ssh_arguments = "-o ConnectTimeout=2"
38:cleanup = true
40:[git]
43:repos = [
44:  "Library/Application Support/PrusaSlicer"
45:]
53:[composer]
57:[pre_commands]
59:"Update /etc/hosts for multipass VMs" = "~/.config/topgrade-etc_hosts"
62:[commands]
63:"01 Update downloaded apps" = "~/bin/update_downloaded-apps.sh"
64:"12 Create/update ~/.ssh/config.known_hosts" = "~/.config/topgrade-ssh_config.known-hosts"
65:"13 Create/update ~/.ssh/pssh_lists" = "~/.config/topgrade-pssh_lists"
66:"14 Clone/update Git repositories (with MyRepos)" = "~/.config/topgrade-myrepos"
67:"20 Sync home with git" = "~/bin/sync-home-with-git.sh"
68:"21 Sync Source/xxx to Git/xxx" = "rsync -aHxu --times --omit-dir-times --omit-link-times --safe-links --no-devices --no-specials --human-readable --info=name,copy,del,symsafe --exclude .git --exclude .history --delete ~/Source/xxx ~/Git/xxx/"
69:"22 Sync Source/yyy to Git/yyy" = "rsync -aHxu --times --omit-dir-times --omit-link-times --safe-links --no-devices --no-specials --human-readable --info=name,copy,del,symsafe --exclude .git --exclude .history --delete ~/Source/yyy ~/Git/yyy/"
70:"50 Backup" = "~/bin/backup-user"
71:"90 Update & backup home routers" = "~/.config/topgrade-update_routers"
72:"99 Status of Git repositories (with MyRepo)" = "~/.config/topgrade-myrepos --status"
74:[brew]
75:greedy_cask = true
77:[linux]
90:[windows]
100:[npm]
104:[firmware]
108:[flatpak]

Additional Details

Operation System/Version

% system_profiler SPSoftwareDataType SPHardwareDataType Software:

System Software Overview:

System Version: macOS 13.5.2 (22G91) Kernel Version: Darwin 22.6.0 Boot Volume: Macintosh HD Boot Mode: Normal Computer Name: User Name: Reini () Secure Virtual Memory: Enabled System Integrity Protection: Enabled Time since boot: 38 minutes, 36 seconds

Hardware:

Hardware Overview:

  Model Name: MacBook Pro
  Model Identifier: MacBookPro18,3
  Model Number: Z15G0008DD/A
  Chip: Apple M1 Pro
  Total Number of Cores: 8 (6 performance and 2 efficiency)
  Memory: 32 GB
  System Firmware Version: 8422.141.2
  OS Loader Version: 8422.141.2
  Serial Number (system): ***
  Hardware UUID: ***
  Provisioning UDID: ***
  Activation Lock Status: Enabled

Installation

with homebrew (brew)
Topgrade version (topgrade -V)

% topgrade -V
Topgrade 12.0.2

Verbose Output (`topgrade -v`)

Not possible at the moment, as there is no other system update available now ...

DucretJe commented 1 year ago

Same here

SteveLauC commented 1 year ago

Here are the commands Topgrade use to update macOS:

$ softwareupdate --list              # check for updates
$ softwareupdate --install --all     # install updates

Would you like to try them to see if the issue still exists?

Not possible at the moment, as there is no other system update available now ...

You can try them when there is a system update available:)

reini-1 commented 1 year ago

I assume you have to enter the user's password here as you have to do it with system settings gui.

I will test this with the next update (if I don't forget it...)

DucretJe commented 1 year ago

On a Topgrade 12.0.2 iMac Pro (Intel) 13.4.1

So as mentioned before I have same behaviour than raised by OP:

── 16:28:51 - macOS system update ──────────────────────────────────────────────
Finding available software
A system update is available. Do you wish to install it? (y)es/(N)o
Software Update Tool

Finding available software
Downloading macOS Ventura 13.5.2
Downloading: 100.00%
Downloaded: macOS Ventura 13.5.2

softwareupdate --list
Software Update Tool

Finding available software
Software Update found the following new or updated software:
* Label: macOS Ventura 13.5.2-22G91
    Title: macOS Ventura 13.5.2, Version: 13.5.2, Size: 1018917KiB, Recommended: YES, Action: restart,

softwareupdate --install --all
Software Update Tool

Finding available software
Downloading macOS Ventura 13.5.2

Downloaded: macOS Ventura 13.5.2

Same behaviour with the softwareupdate tool, so maybe a macOS issue...

SteveLauC commented 1 year ago

Since this issue can be reproduced with the underlying commands Topgrade uses, I will label it upstream.

BTW, what is the version of the softwareupdate command?

reini-1 commented 1 year ago

I think sudo is needed to install the updates, as many websites e.g. https://iboysoft.com/news/update-mac-from-terminal.html tell me.

SteveLauC commented 1 year ago

Friendly ping @DucretJe, would you like to give this a test to see if sudo would make it work here?

$ softwareupdate --list             
$ sudo softwareupdate --install --all

DucretJe commented 1 year ago

I didn’t saw any error due to the lack of permissions so I did not thought of that -.- I don’t have any outdated machine anymore, will do next time!

reini-1 commented 1 year ago

✓ macOS-13.5.2> reini@mbp ~ % softwareupdate --list
Software Update Tool

Finding available software
Software Update found the following new or updated software:
* Label: Command Line Tools for Xcode-15.0
    Title: Command Line Tools for Xcode, Version: 15.0, Size: 721962KiB, Recommended: YES,
✓ macOS-13.5.2> reini@mbp ~ % sudo softwareupdate --install --all
Password:
Software Update Tool

Finding available software

Downloaded Command Line Tools for Xcode
Installing Command Line Tools for Xcode
Done with Command Line Tools for Xcode
Done.
✓ macOS-13.5.2> reini@mbp ~ % date
Mi 20 Sep 2023 14:03:28 CEST

For a xcode update this looks good, but did not try without sudo...

reini-1 commented 1 year ago

✓ macOS-13.5.2> reini@mbp ~ % softwareupdate --list
Software Update Tool

Finding available software
Software Update found the following new or updated software:
* Label: macOS Ventura 13.6-22G120
        Title: macOS Ventura 13.6, Version: 13.6, Size: 988023KiB, Recommended: YES, Action: restart,
✓ macOS-13.5.2> reini@mbp ~ % sudo softwareupdate --install --all
Software Update Tool

Finding available software
Downloading macOS Ventura 13.6
Password:

Downloaded: macOS Ventura 13.6

... and waiting ...

A logout did not continue the update process and reboot the machine.

I think --restart and/or --force has to be added:

-R | --restart
        Automatically restart (or shut down) if required to complete installation. If the
        user invoking this tool is logged in then macOS will attempt to quit all
        applications, logout, and restart. If the user is not logged in, macOS will trigger a
        forced reboot if necessary. If you wish to always perform a forced reboot, pass -f
        (--force).

When using sudo softwareupdate --install --all --restart --force a automatic restart is done without a warning before. Maybe there is a warning/question without --force but I cannot test this again at the moment as there is no other update and I don't have another machine ...

It would also be possible to only download the update and not install it and tell the user that he/she should do it with "system settings" or warn the user that the machine will reboot automatically or ...

I also did not know if sudo is necessary or not.

Or maybe dependent on the "Action: restart" ask the user if a automatic restart should be done or the update should only be downloaded or make this configurable or ...

DucretJe commented 1 year ago

On MacOS Ventura 13.6

Trid without sudo:

softwareupdate --install --all --restart --force
Passing --restart requires root privilege.

Well, now we know.

sudo softwareupdate --install --all --restart --force
Password:
Software Update Tool

Finding available software
Downloading macOS Ventura 13.6
Downloading: 100.00%
Downloaded: macOS Ventura 13.6
Attempting to quit apps: (
    "com.apple.Safari.SafariQuickLookPreview",
    "com.apple.Safari"
)
Successfully quit all apps

Downloading Safarizsh: terminated  sudo softwareupdate --install --all --restart --force

It worked

SteveLauC commented 1 year ago

Ok, we now know that option --restart requires sudo, but we don't know if the lack of this option is the root cause of this issue.

There is an online man page for this command, but it is kinda outdated, would you like to give me the output of

$ man softwareupdate

SteveLauC commented 1 year ago

Found this new man page today:

softwareupdate requires admin authentication for all commands except --list. If you run softwareupdate as a normal admin user, you will be prompted for a password where required. Alternatively, you can run softwareupdate as root and avoid all further authentication prompts.

Seems like sudo is needed?

If you run softwareupdate as a normal admin user, you will be prompted for a password where required.

Did you guys get a prompt for entering your password when using it without a sudo?

DucretJe commented 11 months ago

man softwareupdate

Details

softwareupdate(8) System Manager's Manual softwareupdate(8) NAME softwareupdate – system software update tool SYNOPSIS softwareupdate command [args ...] DESCRIPTION Software Update checks for new and updated versions of your software based on information about your computer and current software. Invoke softwareupdate by specifying a command followed by zero or more args. softwareupdate requires admin authentication for all commands except --list. If you run softwareupdate as a normal admin user, you will be prompted for a password where required. Alternatively, you can run softwareupdate as root and avoid all further authentication prompts. The following commands are available: -l | --list List all available updates. -i | --install Each update specified by args is downloaded and installed. args can be one of the following: -r | --recommended All updates that are recommended for your system. These are prefixed with a * character in the --list output. --os-only Only macOS updates --safari-only Only safari updates -R | --restart Automatically restart (or shut down) if required to complete installation. If the user invoking this tool is logged in then macOS will attempt to quit all applications, logout, and restart. If the user is not logged in, macOS will trigger a forced reboot if necessary. If you wish to always perform a forced reboot, pass -f (--force). -a | --all All updates that are applicable to your system, including those non-recommended ones, which are prefixed with a - character in the --list output. (Non-recommended updates are uncommon in any case.) item ... One or more specified updates. The --list output shows the item names you can specify here, prefixed by the * or - characters. See EXAMPLES. --stdinpass Password to authenticate as an owner. Apple Silicon only. --user Local username to authenticate as an owner. Apple Silicon only. --list-full-installers List the available macOS Installers. --fetch-full-installer Install the latest recommended macOS Installer. Use --full-installer-version to specify the version of macOS to install. ie. --full-installer-version 10.15. Use --launch-installer to launch the installer automatically after it has been downloaded --install-rosetta Install Rosetta. Only applies to Apple silicon Macs. Pass --agree-to-license to agree to the software license agreement without any user interaction. -d | --download -h | --help Print command usage. EXAMPLES The following examples are shown as given to the shell: softwareupdate --list Software Update Tool Finding available software Software Update found the following new or updated software: * Label: MacBookAirEFIUpdate2.4-2.4 Title: MacBook Air EFI Firmware Update, Version: 2.4, Size: 3817K, Recommended: YES, Action: restart, * Label: ProAppsQTCodecs-1.0 Title: ProApps QuickTime codecs, Version: 1.0, Size: 968K, Recommended: YES, sudo softwareupdate --install ProAppsQTCodecs-1.0 Software Update Tool Finding available software Downloading ProApps QuickTime codecs Downloaded ProApps QuickTime codecs Installing ProApps QuickTime codecs Done with ProApps QuickTime codecs Done. sudo softwareupdate --schedule Automatic check is on Mac OS X September 11, 2012 Mac OS X

DucretJe commented 11 months ago

Tried to run the previous command without sudo and without --restart will indeed prompt for password.

softwareupdate --install --all --force
Software Update Tool

Finding available software
Downloading macOS Sonoma 14.2
Password:
Downloading: 100.00%
Downloaded: macOS Sonoma 14.2

Downloaded Command Line Tools for Xcode
Installing Command Line Tools for Xcode
Done with Command Line Tools for Xcode
Done.
Downloaded: macOS Sonoma 14.2

Worked

Upgrade from 14.1.2 -> 14.2

SteveLauC commented 11 months ago

Ok then, I think we should not use the --force option as it will require a reboot on every update, we only want to reboot when needed, so let's use the --restrat flag. Since --restart requires sudo, let's do

$ sudo softwareupdate --install --all --restart

DucretJe commented 11 months ago

If we put the --restart flag, wouldn't it restart in the middle of the topgrade execution?

I can try to build your branch on a Mac if it helps

SteveLauC commented 11 months ago

If we put the --restart flag, wouldn't it restart in the middle of the topgrade execution?

I think so, if the updated packages need a reboot, then macOS will do it.

And this seems to be unavoidable as we are not clear about what packages need a reboot

SteveLauC commented 11 months ago

Well, we should probably not merge that PR because I can see that users will complain Topgrade reboots their system without permission...

DucretJe commented 11 months ago

softwareupdate --install --all --force installs the updates (asking for a password) but won't restart, I think it would be a good alternative

SteveLauC commented 11 months ago

softwareupdate --install --all --force installs the updates (asking for a password) but won't restart, I think it would be a good alternative

Per the man page, this option seems to force a reboot every time?

If you wish to always perform a forced reboot, pass -f (--force).

DucretJe commented 10 months ago

softwareupdate --install --all --force installs the updates (asking for a password) but won't restart, I think it would be a good alternative

Per the man page, this option seems to force a reboot every time?

If you wish to always perform a forced reboot, pass -f (--force).

Sorry missed the notification, it didn't restart on my side when I tested it. I'll retry on different machines for the next update.

SteveLauC commented 10 months ago

Sigh, fxxk apple, I can do nothing with this issue for now, unless someone who is VERY familiar with this undocumented command tells me what should be involved.

topgrade-rs / topgrade