search

blacktop / ipsw

iOS/macOS Research Swiss Army Knife
https://blacktop.github.io/ipsw
MIT License
1.81k stars 141 forks source link

Failure to extract DSC without first mounting sys #544

Closed mc-17 closed 2 weeks ago

mc-17 commented 3 weeks ago

What happened?

Tried to extract DSC from iOS 18 beta 7

e41a72d49fd246fed0110e4342672bafc709e76c - IPSW hash

[IPSW Info]
===========
Version        = 18.0
BuildVersion   = 22A5346a
OS Type        = Development
FileSystem     = 090-28430-084.dmg.aea
SystemOS       = 090-29322-099.dmg.aea
AppOS          = 090-28378-122.dmg
RestoreRamDisk = [090-27766-125.dmg 090-27884-122.dmg]

Devices
-------

iPhone 15 Pro
 > iPhone16,1_D83AP_22A5346a
   - TimeStamp: 14 Aug 2024 21:44:57 PDT
   - KernelCache: kernelcache.release.iphone16
   - CPU: A17 Pro (), ID: t8130
   - BootLoaders
       * iBEC.d83.RELEASE.im4p
       * iBoot.d83.RELEASE.im4p
       * iBSS.d83.RELEASE.im4p
       * LLB.d83.RELEASE.im4p
       * sep-firmware.d83.RELEASE.im4p

by running ipsw extract -d <ipsw>, and it fails.

If I instead run ipsw mount sys <ipsw> and then this command, it succeeds:

michael@michaels-MBP ipsws % ipsw extract -d beta7.ipsw
   • Extracting dyld_shared_cache
      • Mounting DMG 090-29322-099.dmg
         • failed to walk /tmp/090-29322-099.dmg.mount: lstat /tmp/090-29322-099.dmg.mount: no such file or directory
Usage:
  ipsw extract <IPSW/OTA | URL> [flags]

Aliases:
  extract, e, ex

Flags:
      --device string           Device to extract kernel for (e.g. iPhone10,6)
      --dmg string              Extract DMG file (app, sys, fs)
      --driverkit               Extract DriverKit dyld_shared_cache
      --dtree                   Extract DeviceTree
  -d, --dyld                    Extract dyld_shared_cache
  -a, --dyld-arch stringArray   dyld_shared_cache architecture to extract
  -x, --exclave                 Extract Exclave Bundle
      --fcs-key                 Extract AEA1 DMG fcs-key pem files
  -f, --files                   Extract File System files
      --flat                    Do NOT perserve directory structure when extracting
  -h, --help                    help for extract
      --iboot                   Extract iBoot
      --insecure                do not verify ssl certs
  -j, --json                    Output extracted paths as JSON
      --kbag                    Extract Im4p Keybags
  -k, --kernel                  Extract kernelcache
  -o, --output string           Folder to extract files to
  -p, --pattern string          Extract files that match regex
      --pem-db string           AEA pem DB JSON file
      --proxy string            HTTP/HTTPS proxy
  -r, --remote                  Extract from URL
      --sep                     Extract sep-firmware
      --sptm                    Extract SPTM and TXM Firmwares
      --sys-ver                 Extract SystemVersion

Global Flags:
      --color           colorize output
      --config string   config file (default is $HOME/.config/ipsw/config.yaml)
      --no-color        disable colorize output
  -V, --verbose         verbose output

   ⨯ failed to find dyld_shared_cache(s) in DMG: 090-29322-099.dmg
michael@michaels-MBP ipsws % ipsw mount sys beta7.ipsw &
[1] 21577
michael@michaels-MBP ipsws % ipsw mount sys beta7.ipsw &   • Mounted sys DMG 090-29322-099.dmg
      • Press Ctrl+C to unmount '/tmp/090-29322-099.dmg.mount' ...
extract -d beta7.ipsw 
   • Extracting dyld_shared_cache
      • Mounting DMG 090-29322-099.dmg
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.01
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.02
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.03
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.04
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.05
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.06
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.07
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.08
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.09
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.10
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.11
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.12
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.13
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.14
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.15
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.16
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.17
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.18
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.19
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.20
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.21
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.22
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.23
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.24
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.25
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.26
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.27
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.28
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.29
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.30
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.31.dylddata
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.32.dyldlinkedit
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.33
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.34
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.35
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.36
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.37
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.38
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.39
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.40
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.41
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.42
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.43
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.44
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.45
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.46
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.47
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.48
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.49
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.50
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.51
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.52
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.53
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.54
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.55
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.56
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.57
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.58
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.59
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.60
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.61.dylddata
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.62.dyldlinkedit
      • Created 22A5346a__iPhone16,1/dyld_shared_cache_arm64e.symbols
michael@michaels-MBP ipsws %

I assume it's some sort of race from mounting to then looking for the DSC. Weirdly enough, I don't see this with beta 8 (hash ae42ef0e7ffda7a625a1d5f3e578be1cbcec5633)

How can we reproduce this?

  1. Download iPhone 15 Pro 18.0b7
  2. Run ipsw extract -d <ipsw>
  3. It fails
  4. Mount the DMG manually with ipsw mount sys <ipsw> and then the command
  5. It works

ipsw version

Version: 3.1.538, BuildCommit: 515f4efbde44621df98ca24bc62cbb616b086473

Search

Code of Conduct

Additional context

No response

mc-17 commented 3 weeks ago

Weirdly enough, I don't see this with the following beta7 IPSW

[IPSW Info]
===========
Version        = 18.0
BuildVersion   = 22A5346a
OS Type        = Development
FileSystem     = 090-28877-084.dmg.aea
SystemOS       = 090-28396-099.dmg.aea
AppOS          = 090-28378-122.dmg
RestoreRamDisk = [090-27766-125.dmg 090-27884-122.dmg]

Devices
-------

iPhone 13 Pro Max
 > iPhone14,3_D64AP_22A5346a
   - TimeStamp: 14 Aug 2024 21:44:37 PDT
   - KernelCache: kernelcache.release.iphone14
   - CPU: A15 Bionic (ARMv8.5-A), ID: t8110
   - BootLoaders
       * iBEC.d64.RELEASE.im4p
       * iBoot.d64.RELEASE.im4p
       * iBSS.d64.RELEASE.im4p
       * LLB.d64.RELEASE.im4p
       * sep-firmware.d64.RELEASE.im4p

The problem IPSW is:

https://updates.cdn-apple.com/2024SummerSeed/fullrestores/062-64346/BB6E07F7-70FE-4E42-BE5C-B69FE7CF4243/iPhone16,1_18.0_22A5346a_Restore.ipsw
mc-17 commented 3 weeks ago

This might be a macOS 15 thing - I find that hdiutil detach <mount> -force will make it work, perhaps flipping the force flag on the unmount might resolve this.

blacktop commented 3 weeks ago

Ya I've seen dmgs get stuck occasionally or if you interrupt it and it'll put the already extracted DMG in a weird state.

I thinking trying to force might work. I thought I already was? Or changed it back to not force for some reason that I cannot remember.

However, since I am constantly testing the same, IPSWs over and over again, sometimes with crashes. I get these stuck DMGs is pretty often.

I always delete them if they're on the file system still. And then manually unmount to fix.

Which definitely is not a great solution for end users if they're seeing it happened too

blacktop commented 3 weeks ago

https://github.com/blacktop/ipsw/blob/6f0468dc705ad8d64b6d9c2a8c733dc54a6b6447/internal/utils/macos.go#L502

blacktop commented 3 weeks ago

added force to the call in Extract, I probably should always force but again, I feel like I was, and then decided to not make that the default? 🤷

https://github.com/blacktop/ipsw/commit/7480ed5381413f457e52fbbcbc3ba51edfed4f42

blacktop commented 3 weeks ago

pushed an new release, let me know if it made a difference for you

mc-17 commented 3 weeks ago

pushed an new release, let me know if it made a difference for you

This does appear to work, thanks! However it's also occurring in other places (such as ipsw diff when extacting DSCs, with the same ipsw mount workaround fixing it).

Might it be worth flipping the force flag default (at least for RO images perhaps if you want to play it safe)?

blacktop commented 3 weeks ago

we'll see how this fairs ;)

https://github.com/blacktop/ipsw/commit/a1e10bfaabf9ab38f237ff5568eeee7eb9f808bf

blacktop commented 2 weeks ago

feel free to open again, if the issue isn't resolved