search

arandomdev / DyldExtractor

Extract Binaries from Apple's Dyld Shared Cache
MIT License
404 stars 40 forks source link

iOS 15 new shared cache format. #15

Closed 0cyn closed 2 years ago

0cyn commented 3 years ago

latest revision has completely changed the format of the shared cache which may end up requiring a rewrite or seperate project to tackle. Here's a spot to compile info on the topic.

arandomdev commented 3 years ago

Okay, I believe this was introduced in iOS 15 beta. The Dyld Shared Cache (DSC) seems to be split into separate files, i.e.

Looking at the header of each of these, we can find out a few things.

dyld_shared_cache_arm64 ``` dyld_shared_cache_arm64 magic: b'dyld_v1 arm64' mappingOffset: 456 mappingCount: 1 imagesOffset: 0 imagesCount: 0 dyldBaseAddress: 0 codeSignatureOffset: 1070710784 codeSignatureSize: 8372224 slideInfoOffsetUnused: 0 slideInfoSizeUnused: 0 localSymbolsOffset: 0 localSymbolsSize: 0 uuid: [83, 218, 0, 137, 88, 97, 52, 244, 172, 33, 253, 129, 13, 125, 99, 19] cacheType: 1 branchPoolsOffset: 0 branchPoolsCount: 0 accelerateInfoAddr: 0 accelerateInfoSize: 0 imagesTextOffset: 75832 imagesTextCount: 2352 patchInfoAddr: 8667632592 patchInfoSize: 1361752 otherImageGroupAddrUnused: 0 otherImageGroupSizeUnused: 0 progClosuresAddr: 0 progClosuresSize: 0 progClosuresTrieAddr: 0 progClosuresTrieSize: 0 platform: 2 formatVersion: 0 dylibsExpectedOnDisk: 0 simulator: 0 locallyBuiltCache: 0 builtFromChainedFixups: 0 padding: 0 sharedRegionStart: 6442450944 sharedRegionSize: 2236121088 maxSlide: 113328128 dylibsImageArrayAddr: 0 dylibsImageArraySize: 0 dylibsTrieAddr: 8667529216 dylibsTrieSize: 103376 otherImageArrayAddr: 0 otherImageArraySize: 0 otherTrieAddr: 0 otherTrieSize: 0 mappingWithSlideOffset: 512 mappingWithSlideCount: 1 ```
dyld_shared_cache_arm64.1 ``` dyld_shared_cache_arm64.1 magic: b'dyld_v1 arm64' mappingOffset: 456 mappingCount: 1 imagesOffset: 0 imagesCount: 0 dyldBaseAddress: 0 codeSignatureOffset: 586416128 codeSignatureSize: 4587520 slideInfoOffsetUnused: 0 slideInfoSizeUnused: 0 localSymbolsOffset: 0 localSymbolsSize: 0 uuid: [158, 168, 231, 25, 184, 22, 50, 47, 187, 173, 79, 221, 157, 76, 52, 209] cacheType: 1 branchPoolsOffset: 0 branchPoolsCount: 0 accelerateInfoAddr: 0 accelerateInfoSize: 0 imagesTextOffset: 75832 imagesTextCount: 2352 patchInfoAddr: 0 patchInfoSize: 0 otherImageGroupAddrUnused: 0 otherImageGroupSizeUnused: 0 progClosuresAddr: 0 progClosuresSize: 0 progClosuresTrieAddr: 0 progClosuresTrieSize: 0 platform: 2 formatVersion: 0 dylibsExpectedOnDisk: 0 simulator: 0 locallyBuiltCache: 0 builtFromChainedFixups: 0 padding: 0 sharedRegionStart: 7513161728 sharedRegionSize: 0 maxSlide: 0 dylibsImageArrayAddr: 0 dylibsImageArraySize: 0 dylibsTrieAddr: 0 dylibsTrieSize: 0 otherImageArrayAddr: 0 otherImageArraySize: 0 otherTrieAddr: 0 otherTrieSize: 0 mappingWithSlideOffset: 512 mappingWithSlideCount: 1 ```
dyld_shared_cache_arm64.2 ``` dyld_shared_cache_arm64.2 magic: b'dyld_v1 arm64' mappingOffset: 456 mappingCount: 4 imagesOffset: 0 imagesCount: 0 dyldBaseAddress: 0 codeSignatureOffset: 276037632 codeSignatureSize: 2162688 slideInfoOffsetUnused: 0 slideInfoSizeUnused: 0 localSymbolsOffset: 0 localSymbolsSize: 0 uuid: [159, 214, 128, 158, 37, 36, 55, 108, 135, 193, 111, 161, 170, 112, 69, 63] cacheType: 1 branchPoolsOffset: 0 branchPoolsCount: 0 accelerateInfoAddr: 0 accelerateInfoSize: 0 imagesTextOffset: 76168 imagesTextCount: 2352 patchInfoAddr: 0 patchInfoSize: 0 otherImageGroupAddrUnused: 0 otherImageGroupSizeUnused: 0 progClosuresAddr: 0 progClosuresSize: 0 progClosuresTrieAddr: 0 progClosuresTrieSize: 0 platform: 2 formatVersion: 0 dylibsExpectedOnDisk: 0 simulator: 0 locallyBuiltCache: 0 builtFromChainedFixups: 0 padding: 0 sharedRegionStart: 8099577856 sharedRegionSize: 0 maxSlide: 0 dylibsImageArrayAddr: 0 dylibsImageArraySize: 0 dylibsTrieAddr: 0 dylibsTrieSize: 0 otherImageArrayAddr: 0 otherImageArraySize: 0 otherTrieAddr: 0 otherTrieSize: 0 mappingWithSlideOffset: 680 mappingWithSlideCount: 4 ```
dyld_shared_cache_arm64.3 ``` dyld_shared_cache_arm64.3 magic: b'dyld_v1 arm64' mappingOffset: 456 mappingCount: 2 imagesOffset: 0 imagesCount: 0 dyldBaseAddress: 0 codeSignatureOffset: 202293248 codeSignatureSize: 1589248 slideInfoOffsetUnused: 0 slideInfoSizeUnused: 0 localSymbolsOffset: 0 localSymbolsSize: 0 uuid: [245, 34, 192, 200, 68, 6, 50, 231, 187, 94, 165, 39, 97, 46, 114, 21] cacheType: 1 branchPoolsOffset: 0 branchPoolsCount: 0 accelerateInfoAddr: 0 accelerateInfoSize: 0 imagesTextOffset: 75944 imagesTextCount: 2352 patchInfoAddr: 0 patchInfoSize: 0 otherImageGroupAddrUnused: 0 otherImageGroupSizeUnused: 0 progClosuresAddr: 0 progClosuresSize: 0 progClosuresTrieAddr: 0 progClosuresTrieSize: 0 platform: 2 formatVersion: 0 dylibsExpectedOnDisk: 0 simulator: 0 locallyBuiltCache: 0 builtFromChainedFixups: 0 padding: 0 sharedRegionStart: 8476278784 sharedRegionSize: 0 maxSlide: 0 dylibsImageArrayAddr: 0 dylibsImageArraySize: 0 dylibsTrieAddr: 0 dylibsTrieSize: 0 otherImageArrayAddr: 0 otherImageArraySize: 0 otherTrieAddr: 0 otherTrieSize: 0 mappingWithSlideOffset: 568 mappingWithSlideCount: 2 ```
dyld_shared_cache_arm64.symbols ``` dyld_shared_cache_arm64.symbols magic: b'dyld_v1 arm64' mappingOffset: 456 mappingCount: 1 imagesOffset: 0 imagesCount: 0 dyldBaseAddress: 0 codeSignatureOffset: 460079104 codeSignatureSize: 3604480 slideInfoOffsetUnused: 0 slideInfoSizeUnused: 0 localSymbolsOffset: 16384 localSymbolsSize: 460062720 uuid: [44, 127, 111, 44, 107, 123, 62, 64, 173, 193, 47, 206, 7, 168, 149, 236] cacheType: 0 branchPoolsOffset: 0 branchPoolsCount: 0 accelerateInfoAddr: 0 accelerateInfoSize: 0 imagesTextOffset: 0 imagesTextCount: 0 patchInfoAddr: 0 patchInfoSize: 0 otherImageGroupAddrUnused: 0 otherImageGroupSizeUnused: 0 progClosuresAddr: 0 progClosuresSize: 0 progClosuresTrieAddr: 0 progClosuresTrieSize: 0 platform: 2 formatVersion: 0 dylibsExpectedOnDisk: 0 simulator: 0 locallyBuiltCache: 0 builtFromChainedFixups: 0 padding: 0 sharedRegionStart: 0 sharedRegionSize: 0 maxSlide: 0 dylibsImageArrayAddr: 0 dylibsImageArraySize: 0 dylibsTrieAddr: 0 dylibsTrieSize: 0 otherImageArrayAddr: 0 otherImageArraySize: 0 otherTrieAddr: 0 otherTrieSize: 0 mappingWithSlideOffset: 512 mappingWithSlideCount: 1 ```

Basically different elements are separated, for example, all the local symbols are stored in the ".symbols" file. Also by examining the dyld source code, it's likely that the mapping info (located with mappingOffset) is placed directly after the header. Currently, the size of the header is 320 bytes, but the mappingOffset is set to 456, which means that we are missing some fields in the new header.

Unfortunately, until Apple open sources their new format, It would be hard to add support for this.

blacktop commented 3 years ago

Got 3 new header fields so far - https://github.com/blacktop/ipsw/blob/master/hack/extras/Dyld.bt (14 more to go)

arandomdev commented 3 years ago

Interesting! Did you disassemble dyld to get the size of the new fields? Until the actual release of the code, I guess I can look further into it.

blacktop commented 3 years ago

I was going to do that eventually, I am mostly using 010 Editor to jump around and make guesses at what the fields are, if they fields made sense as uint64 or uint32 etc is a try and guess procedure right now, but it got me 3 fields so far ;) There are so MANY new fields, this seems like a pretty big change, but it is very strange that they would move imagesOffset and imagesCount to the bottom and still keep the old field around?? An associate on twitter took the linkers WWDC 2021 lab and they said the in-memory cache will always look the same, but the on-disk might change (meaning that this new format is in flux??) 🤷

arandomdev commented 3 years ago

As for keeping the old field around, I think it's for some cross-compatibility so that older dyld loaders could still make sense of newer caches. Like it would be better for an older loader to read the imageOffset as 0 instead of a new value meant for a different field. But given apple's method of bundling both the loader and the cache together in the IPSW file, it wouldn't make too much of a difference.

Also, it doesn't make sense that they would just move the dyld_cache_image_info structure without some reason. Maybe they did something similar to dyld_cache_mapping_info and dyld_cache_mapping_and_slide_info, where this new image info structure contains more fields.

If possible can you link the Twitter conversation or elaborate more on it?

blacktop commented 3 years ago

https://twitter.com/_saagarjha/status/1402345391706677259

blacktop commented 3 years ago

got another field: (and correct name for "numSplits"

<SNIP>
    uint32 numSubCaches;     // number of dyld_shared_cache .1,.2,.3 files
    Uuid symbolSubCacheUUID; // unique value for .symbols sub-cache
<SNIP>
arandomdev commented 3 years ago

Nice! how are you getting these field names?

blacktop commented 3 years ago

RE of dsc_extractor.bundle

There is also a comparison of subcache UUIDs at the uint32 right above the numSubCaches field, but I am confused if it is a "partial UUID?" or just an ID? but there are all the same for the dsc and the dsc.1, dsc.2 etc. but it refers to it as a UUID which needs 16bytes ??

blacktop commented 3 years ago

Got a few more fields, but they are kinda strange, this format feels un-finalized 🤷

arandomdev commented 3 years ago

It might be a beta thing. They might not adopt this format for early iOS 15 releases.

blacktop commented 2 years ago

I've been meaning to blog about my findings for a while, but I've realized I write english like a complete moron. In the mean time I hope this makes sense to people?

https://github.com/NationalSecurityAgency/ghidra/pull/3666#issuecomment-976151215

arandomdev commented 2 years ago

This is really helpful, I’ll see what I can do with DyldExtractor when I get the chance.

arandomdev commented 2 years ago

I uploaded a beta branch here. It contains all the fixes for the new format, I'm just doing a lot of bug fixes now.

arandomdev commented 2 years ago

iOS 15 caches are now supported with version 2.0.0!