search

XcodesOrg / xcodes

The best command-line tool to install and switch between multiple versions of Xcode.
MIT License
3.6k stars 120 forks source link

Support runtime downloading/installing #230

Closed StevenSorial closed 1 year ago

StevenSorial commented 1 year ago

This PR builds on #223, and adds a command for installing runtimes xcodes runtimes install 'iOS 14.5'

Downloading will require an Apple account for new platforms released in 2022, it's not needed for older platforms.

Installing is still a WIP

StevenSorial commented 1 year ago

@MattKiazyk The PR is still a WIP but I may need some help regarding Aria :)

I rewrote the Downloader to be shared for downloading Xcodes and runtimes, For some reason, progress is not reported when downloading runtimes using Aria, but it's working fine when downloading Xcodes.

MattKiazyk commented 1 year ago

@StevenMagdy Haven't looked much at the PR but curious about:

Downloading will require an Apple account for new platforms released in 2022

I'm pretty sure we are free and clear of that these days, using the new downloadADCAuth first https://github.com/RobotsAndPencils/xcodes/blob/main/Sources/XcodesKit/URLRequest%2BApple.swift#L36

StevenSorial commented 1 year ago

I will investigate more, but Isn't that only for XcodeReleases?

MattKiazyk commented 1 year ago

The loginIfNeeded() is needed for Apple data source to get the Xcode download list. It is not required (if using the downloadADCAuth) for downloading the final xip of Xcode.

So similarly for the simulators, we can get that download path from a public endpoint. So for the final download we just need to make sure we've called the downloadADCAuth to set the cookie headers (which get passed through to urlSession or aria2).

StevenSorial commented 1 year ago

Done. Thanks for the info. Please take a look at the progress issue when you have time 🙏🏼

StevenSorial commented 1 year ago

@MattKiazyk Also fixed #242

StevenSorial commented 1 year ago

The PR now even handles a theoretical edge case where two builds of the same OS are installed, it will add a build number to differentiate the two:

iOS 14.5
iOS 15.5 (build1)
iOS 15.5 (build2)
iOS 16.0
oonoo commented 1 year ago

First I would like to say I am REALLY looking forward to this feature. That would enable us to use xcodes for provisioning our CI/CD computers. And sorry if this is a spam message.

I tested the branch to check if I can install older runtimes with it. But somehow it fails. This is my output:

sudo xcodes runtimes install "iOS 14.5"
Alias tip: _ xcodes runtimes install "iOS 14.5"

Downloading with aria2 (/opt/homebrew/bin/aria2c)
Downloading Runtime iOS 14.5: 99%
Mounting DMG
Installing Runtime
Error: Failed executing: `/usr/sbin/installer -pkg /Users/MY_NAME/Library/Caches/com.robotsandpencils.xcodes/com.apple.pkg.iPhoneSimulatorSDK14_5.pkg -target /` (1).

Maybe it is just some wrong configuration on my computer? Or is this not supposed to work as I assume?

I have Xcode 16.1 currently selected. Running macOS 12.5.1.

When I open this pkg like this sudo open /Users/SOME_NAME/Library/Caches/com.robotsandpencils.xcodes/com.apple.pkg.iPhoneSimulatorSDK14_5.pkg the Installer GUI opens. When I try to install it that way, there is this output in the log including an error which also lets the installation fail:

Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: @(#)PROGRAM:Install  PROJECT:Install-1000
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: @(#)PROGRAM:Installer  PROJECT:Installer-1118
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Hardware: MacBookPro18,3 @ 0 MHz (x 10), 32768 MB RAM
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Running OS Build: macOS 12.5.1 (21G83)
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: SUDO_GID=20
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: SUDO_UID=501
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: USER=root
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: COMMAND_MODE=unix2003
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: MAIL=/var/mail/root
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: __CFBundleIdentifier=com.apple.installer
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: LC_TERMINAL_VERSION=3.4.16
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: LOGNAME=root
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: SUDO_USER=o.bergob
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: TERM=xterm-256color
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: PATH=/Users/o.bergob/.rbenv/shims:/opt/homebrew/bin:/opt/homebrew/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/munki:/Library/Apple/usr/bin
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: LC_CTYPE=UTF-8
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: SSH_AUTH_SOCK=/private/tmp/com.apple.launchd.NFliEt2Qqd/Listeners
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: HOME=/Users/o.bergob
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: COLORTERM=truecolor
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: SHELL=/bin/sh
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: __CF_USER_TEXT_ENCODING=0x1F5:0x0:0x0
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: LSCOLORS=Gxfxcxdxbxegedabagacad
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: LC_TERMINAL=iTerm2
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: SUDO_COMMAND=/usr/bin/open /Users/o.bergob/Library/Caches/com.robotsandpencils.xcodes/com.apple.pkg.iPhoneSimulatorSDK14_5.pkg
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: COLORFGBG=7;0
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: XPC_SERVICE_NAME=application.com.apple.installer.1152921500312164926.1152921500312164931
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: XPC_FLAGS=0x0
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Env: TMPDIR=/var/folders/48/w36cz21j4wgf24nx1xxkv76c0000gn/T/
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: com.apple.pkg.iPhoneSimulatorSDK14_5  Installation Log
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Opened from: /Users/o.bergob/Library/Caches/com.robotsandpencils.xcodes/com.apple.pkg.iPhoneSimulatorSDK14_5.pkg
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Failed to load specified background image
Nov 16 12:29:25 SOME_NAME-MacBook-Pro Installer[47653]: Top component package (null) not found or understood: treat as unsigned
Nov 16 12:29:31 SOME_NAME-MacBook-Pro Installer[47653]: Administrator authorization granted.
Nov 16 12:29:31 SOME_NAME-MacBook-Pro Installer[47653]: ================================================================================
Nov 16 12:29:31 SOME_NAME-MacBook-Pro Installer[47653]: User picked Standard Install
Nov 16 12:29:31 SOME_NAME-MacBook-Pro Installer[47653]: Choices selected for installation:
Nov 16 12:29:31 SOME_NAME-MacBook-Pro Installer[47653]:     Install: "com.apple.pkg.iPhoneSimulatorSDK14_5"
Nov 16 12:29:31 SOME_NAME-MacBook-Pro Installer[47653]:         com.apple.pkg.iPhoneSimulatorSDK14_5.pkg : com.apple.pkg.iPhoneSimulatorSDK14_5 : 14.5.1.1621461325
Nov 16 12:29:31 SOME_NAME-MacBook-Pro Installer[47653]: ================================================================================
Nov 16 12:29:31 SOME_NAME-MacBook-Pro Installer[47653]: It took 0.00 seconds to summarize the package selections.
Nov 16 12:29:32 SOME_NAME-MacBook-Pro Installer[47653]: -[IFPKGDerivedDocument sortedPackageLocations]: result = (
        "file://localhost"
    )
Nov 16 12:29:32 SOME_NAME-MacBook-Pro Installer[47653]: -[IFDInstallController(Private) _buildInstallPlanReturningError:]: location = file://localhost
Nov 16 12:29:32 SOME_NAME-MacBook-Pro Installer[47653]: -[IFDInstallController(Private) _buildInstallPlanReturningError:]: file://localhost/Users/o.bergob/Library/Caches/com.robotsandpencils.xcodes/com.apple.pkg.iPhoneSimulatorSDK14_5.pkg
Nov 16 12:29:32 SOME_NAME-MacBook-Pro Installer[47653]: Set authorization level to root for session
Nov 16 12:29:32 SOME_NAME-MacBook-Pro Installer[47653]: Packages have been authorized for installation.
Nov 16 12:29:32 SOME_NAME-MacBook-Pro Installer[47653]: Will use PK session
Nov 16 12:29:32 SOME_NAME-MacBook-Pro Installer[47653]: Using authorization level of root for IFPKInstallElement
Nov 16 12:29:58 SOME_NAME-MacBook-Pro Installer[47653]: Starting installation:
Nov 16 12:29:58 SOME_NAME-MacBook-Pro Installer[47653]: Configuring volume "Macintosh HD"
Nov 16 12:29:58 SOME_NAME-MacBook-Pro Installer[47653]: Preparing disk for local booted install.
Nov 16 12:29:58 SOME_NAME-MacBook-Pro Installer[47653]: Free space on "Macintosh HD": 132,63 GB (132625895424 bytes).
Nov 16 12:29:58 SOME_NAME-MacBook-Pro Installer[47653]: Create temporary directory "/var/folders/48/w36cz21j4wgf24nx1xxkv76c0000gn/T//Install.47653ybEoJM"
Nov 16 12:29:58 SOME_NAME-MacBook-Pro Installer[47653]: IFPKInstallElement (1 packages)
Nov 16 12:29:58 SOME_NAME-MacBook-Pro Installer[47653]: Current Path: /System/Library/CoreServices/Installer.app/Contents/MacOS/Installer
Nov 16 12:29:58 SOME_NAME-MacBook-Pro installd[1382]: PackageKit: Adding client PKInstallDaemonClient pid=47653, uid=501 (/System/Library/CoreServices/Installer.app/Contents/MacOS/Installer)
Nov 16 12:29:58 SOME_NAME-MacBook-Pro Installer[47653]: PackageKit: Enqueuing install with framework-specified quality of service (utility)
Nov 16 12:29:58 SOME_NAME-MacBook-Pro installd[1382]: PackageKit: Set reponsibility for install to 47653
Nov 16 12:29:58 SOME_NAME-MacBook-Pro installd[1382]: PackageKit: ----- Begin install -----
Nov 16 12:29:58 SOME_NAME-MacBook-Pro installd[1382]: PackageKit: request=PKInstallRequest <1 packages, destination=/>
Nov 16 12:29:58 SOME_NAME-MacBook-Pro installd[1382]: PackageKit: packages=(
        "PKLeopardPackage <id=com.apple.pkg.iPhoneSimulatorSDK14_5, version=14.5.1.1621461325, url=file://localhost/Users/o.bergob/Library/Caches/com.robotsandpencils.xcodes/com.apple.pkg.iPhoneSimulatorSDK14_5.pkg>"
    )
Nov 16 12:30:28 SOME_NAME-MacBook-Pro installd[1382]: PackageKit: Extracting file://localhost/Users/o.bergob/Library/Caches/com.robotsandpencils.xcodes/com.apple.pkg.iPhoneSimulatorSDK14_5.pkg (destination=/Library/InstallerSandboxes/.PKInstallSandboxManager/45248378-B606-4624-9074-3DFA79B12F7D.activeSandbox/Root/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 14.5.simruntime, uid=0)
Nov 16 12:31:43 SOME_NAME-MacBook-Pro installd[1382]: PackageKit: prevent user idle system sleep
Nov 16 12:31:43 SOME_NAME-MacBook-Pro installd[1382]: PackageKit: suspending backupd
Nov 16 12:31:43 SOME_NAME-MacBook-Pro installd[1382]: PackageKit: Using trashcan path /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T/PKInstallSandboxTrash/45248378-B606-4624-9074-3DFA79B12F7D.sandboxTrash for sandbox /Library/InstallerSandboxes/.PKInstallSandboxManager/45248378-B606-4624-9074-3DFA79B12F7D.activeSandbox
Nov 16 12:31:43 SOME_NAME-MacBook-Pro installd[1382]: PackageKit: Shoving /Library/InstallerSandboxes/.PKInstallSandboxManager/45248378-B606-4624-9074-3DFA79B12F7D.activeSandbox/Root (1 items) to /
Nov 16 12:31:43 SOME_NAME-MacBook-Pro install_monitor[47694]: Temporarily excluding: /Applications, /Library, /System, /bin, /private, /sbin, /usr
Nov 16 12:31:43 SOME_NAME-MacBook-Pro shove[47695]: Error relinking file (primary): /Library/InstallerSandboxes/.PKInstallSandboxManager/45248378-B606-4624-9074-3DFA79B12F7D.activeSandbox/Root/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 14.5.simruntime to /Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 14.5.simruntime, error = 1
    srcPath = /Library/InstallerSandboxes/.PKInstallSandboxManager/45248378-B606-4624-9074-3DFA79B12F7D.activeSandbox/Root/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 14.5.simruntime NSFileOwnerAccountID=0 NSFileSystemFileNumber=76657826 NSFileExtensionHidden=0 NSFileSystemNumber=16777232 NSFileSize=96 NSFileGroupOwnerAccountID=80 NSFileOwnerAccountName=root NSFileCreationDate=2022-11-16 11:30:28 +0000 NSFilePosixPermissions=1021 NSFileType=NSFileTypeDirectory NSFileGroupOwnerAccountName=admin NSFileReferenceCount=3 NSFileModificationDate=2022-11-16 11:30:28 +0000
    dstParentPath = /Library/Developer/CoreSimulator/Profiles/Runtimes NSFileOwnerAccountID=0 NSFileSystemFileNumber=2681196 NSFileExtensionHidden=0 NSFileSystemNumber=16777232 NSFileSize=128 NSFileGroupOwnerAccountID=0 NSFileOwnerAccountName=root NSFileCreationDate=2022-03-24 09:50:39 +0000 NSFilePosixPermissions=493 NSFileType=NSFileTypeDirectory NSFileGroupOwnerAccountName=wheel NSFileReferenceCount=4 NSFileModificationDate=2022-11-08 07:37:27 +0000
Nov 16 12:31:43 SOME_NAME-MacBook-Pro shove[47695]: [src=unknown] /Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 14.5.simruntime: unable to restore flags to destination due to failed lstat (error 2)
Nov 16 12:31:43 SOME_NAME-MacBook-Pro shove[47695]: [source=dir, dst=notExist] failed _relinkFile(/Library/InstallerSandboxes/.PKInstallSandboxManager/45248378-B606-4624-9074-3DFA79B12F7D.activeSandbox/Root/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 14.5.simruntime, /Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 14.5.simruntime): Operation not permitted
    srcPath = /Library/InstallerSandboxes/.PKInstallSandboxManager/45248378-B606-4624-9074-3DFA79B12F7D.activeSandbox/Root/Library/Developer/CoreSimulator/Profiles/Runtimes/iOS 14.5.simruntime NSFileOwnerAccountID=0 NSFileSystemFileNumber=76657826 NSFileExtensionHidden=0 NSFileSystemNumber=16777232 NSFileSize=96 NSFileGroupOwnerAccountID=80 NSFileOwnerAccountName=root NSFileCreationDate=2022-11-16 11:30:28 +0000 NSFilePosixPermissions=1021 NSFileType=NSFileTypeDirectory NSFileGroupOwnerAccountName=admin NSFileReferenceCount=3 NSFileModificationDate=2022-11-16 11:30:28 +0000
    dstParentPath = /Library/Developer/CoreSimulator/Profiles/Runtimes NSFileOwnerAccountID=0 NSFileSystemFileNumber=2681196 NSFileExtensionHidden=0 NSFileSystemNumber=16777232 NSFileSize=128 NSFileGroupOwnerAccountID=0 NSFileOwnerAccountName=root NSFileCreationDate=2022-03-24 09:50:39 +0000 NSFilePosixPermissions=493 NSFileType=NSFileTypeDirectory NSFileGroupOwnerAccountName=wheel NSFileReferenceCount=4 NSFileModificationDate=2022-11-08 07:37:27 +0000
Nov 16 12:31:43 SOME_NAME-MacBook-Pro install_monitor[47694]: Re-included: /Applications, /Library, /System, /bin, /private, /sbin, /usr
Nov 16 12:31:44 SOME_NAME-MacBook-Pro installd[1382]: PackageKit: releasing backupd
Nov 16 12:31:44 SOME_NAME-MacBook-Pro installd[1382]: PackageKit: allow user idle system sleep
Nov 16 12:31:44 SOME_NAME-MacBook-Pro installd[1382]: PackageKit: Install Failed: Error Domain=PKInstallErrorDomain Code=120 "An unexpected error occurred while moving files to the final destination." UserInfo={NSLocalizedDescription=An unexpected error occurred while moving files to the final destination., arguments=(
        "-f",
        "-s",
        "/Library/InstallerSandboxes/.PKInstallSandboxManager/45248378-B606-4624-9074-3DFA79B12F7D.activeSandbox/Root",
        "/"
    ), NSUnderlyingError=0x15b9dd3f0 {Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted"}} {
        NSLocalizedDescription = "An unexpected error occurred while moving files to the final destination.";
        NSUnderlyingError = "Error Domain=NSPOSIXErrorDomain Code=1 \"Operation not permitted\"";
        arguments =     (
            "-f",
            "-s",
            "/Library/InstallerSandboxes/.PKInstallSandboxManager/45248378-B606-4624-9074-3DFA79B12F7D.activeSandbox/Root",
            "/"
        );
    }
Nov 16 12:31:44 SOME_NAME-MacBook-Pro installd[1382]: PackageKit: Cleared responsibility for install from 47653.
Nov 16 12:31:44 SOME_NAME-MacBook-Pro installd[1382]: PackageKit: Cleared permissions on Installer.app
Nov 16 12:31:44 SOME_NAME-MacBook-Pro installd[1382]: PackageKit: Running idle tasks
Nov 16 12:31:44 SOME_NAME-MacBook-Pro Installer[47653]: install:didFailWithError:Error Domain=PKInstallErrorDomain Code=120 "An unexpected error occurred while moving files to the final destination." UserInfo={NSUnderlyingError=0x60000bea50e0 {Error Domain=NSPOSIXErrorDomain Code=1 "Operation not permitted"}, NSLocalizedDescription=An unexpected error occurred while moving files to the final destination., arguments=(
        "-f",
        "-s",
        "/Library/InstallerSandboxes/.PKInstallSandboxManager/45248378-B606-4624-9074-3DFA79B12F7D.activeSandbox/Root",
        "/"
    )}
Nov 16 12:31:44 SOME_NAME-MacBook-Pro installd[1382]: PackageKit: Removing client PKInstallDaemonClient pid=47653, uid=501 (/System/Library/CoreServices/Installer.app/Contents/MacOS/Installer)
Nov 16 12:31:44 SOME_NAME-MacBook-Pro Installer[47653]: Install failed: The Installer encountered an error that caused the installation to fail. Contact the software manufacturer for assistance.
Nov 16 12:31:44 SOME_NAME-MacBook-Pro Installer[47653]: IFDInstallController 3296C80 state = 8
Nov 16 12:31:44 SOME_NAME-MacBook-Pro Installer[47653]: Displaying 'Install Failed' UI.
Nov 16 12:31:44 SOME_NAME-MacBook-Pro Installer[47653]: 'Install Failed' UI displayed message:'The Installer encountered an error that caused the installation to fail. Contact the software manufacturer for assistance.'.
Nov 16 12:31:56 SOME_NAME-MacBook-Pro installd[1382]: PackageKit: Done with sandbox removals
StevenSorial commented 1 year ago

Can you check if your terminal does have Full Disk Access in the macOS preferences? if you are using iTerm, you need to give it the permission explicitly.

oonoo commented 1 year ago

That was the solution! Thanks a lot. :-) Blazing fast runtimes installations with xcodes woohooo!

MattKiazyk commented 1 year ago

@StevenMagdy thanks for all this work - is it ready for another review?

StevenSorial commented 1 year ago

Yes, its ready for a review 👍🏼