Automatically find/calculate offsets in binaries that haven't been manually reversed

[0xdevalias]

As I mentioned on my last PR (https://github.com/beeper/mac-registration-provider/pull/7#issuecomment-1867733658) I've been working on a PoC script that is able to find the offsets automagically (at least across the 13.3.1, 13.5, 14.2 x86 binaries I have tested it against so far):

I tested your script and it finds the correct offsets for 14.0-14.3

Nice, good to know!

I just finished making a little "dump hex at offset in fat binary" helper script too, so should be able to use that to make patterns to match arm automagically as well:

---

I've also been working on a PoC script that is able to find the offsets automagically (at least across the 13.3.1, 13.5, 14.2 x86 binaries I have tested it against so far). I'll probably continue working on it tomorrow, and then hopefully push the code on a PoC repo

This is the repo:

• https://github.com/0xdevalias/poc-re-binsearch

The 'find offsets automagically' script is:

• https://github.com/0xdevalias/poc-re-binsearch/blob/main/find_fat_binary_offsets.py

Some helper scripts for figuring out what to use as the patterns there include:

• https://github.com/0xdevalias/poc-re-binsearch/blob/main/rafind2-fat-binary
• https://github.com/0xdevalias/poc-re-binsearch/blob/main/xxd-fat-binary

You might find some useful or interesting notes here:

• https://github.com/0xdevalias/poc-re-binsearch/blob/main/notes.md

There might be more stuff that is/will end up in that repo that could also be useful; but for now, I think that is the bulk of it.

Originally posted by @0xdevalias in https://github.com/beeper/mac-registration-provider/issues/7#issuecomment-1868266660

The concepts used here could be ported into mac-registration-provider and used to provide support for platform versions where the proper offsets aren't currently known (on a "it should work, but no guarantees" best case effort)

It might also be useful if mac-registration-provider was able to prompt the user to submit a PR/support request/similar with the relevant offsets/etc for their platform if they weren't already known (and were identified with this), and/or suggest how they can share the binary for their platform (along with appropriate version details) so the offsets can be calculated/verified, then added to the main tool.

[0xdevalias]

Added a script that can assist in diffing the hex bytes to create patterns for find_fat_binary_offsets.py / rafind2-fat-binary / rafind2 / etc in https://github.com/0xdevalias/poc-re-binsearch/commit/133c7344c1fcc5d41d828f6ef105b6613df6c6a7 + improved in https://github.com/0xdevalias/poc-re-binsearch/commit/dd231a625518a49fbc1f4cb6379742feba90c2e8:

• https://github.com/0xdevalias/poc-re-binsearch/blob/main/diff_hex.py

---

Using that, I also have patterns that seem to uniquely match the arm offsets for 13.3.1, 13.5, and 14.2.

It seems the arm patterns need to be a lot longer to make them match uniquely, as they were hitting a lot more duplicates in the binaries.

I haven't pushed the patterns I have yet, as I'm wondering if there is an easy way for me to find the minimal length for uniqueness.

[0xdevalias]

Refactored find_fat_binary_offsets.py to add --shortest-patterns feature in https://github.com/0xdevalias/poc-re-binsearch/commit/6978985ebf8d277c7dac2b49990e017ea600ff70

This can be used to optimise the patterns used for finding the offsets to the shortest pattern that still gives a unique match. This should be used against all the test binaries you have, as sometimes a shorter pattern in one breaks the uniqueness for another. In that case, just use the slightly longer one. Repeat until all your test binaries are happy with unique offsets found.

Added unoptimised arm patterns in https://github.com/0xdevalias/poc-re-binsearch/commit/f5471ed5beaeb99e95894cc50ce4c88108c72f62

Optimised the patterns for x86/arm (for the 13.3.1, 13.5, and 14.2 binaries I had to test against) in https://github.com/0xdevalias/poc-re-binsearch/commit/5bf9e9ef98437f26f39c17bdea9db3990767708d

These optimised patterns might need to be adjusted slightly if other binary versions don't work properly with them; but at least for the binaries I have to test against, these seem to work to automagically get the offsets for both x86/arm! 🎉

---

You can see the existing 'known' offsets here:

• https://github.com/beeper/mac-registration-provider/blob/main/nac/offsets.go

And can contrast them against the automagically calculated offsets below to ensure correctness:

13.3.1

```bash ⇒ ./find_fat_binary_offsets.py samples/macos-13.3.1-22E261-ventura-arm64-identityservicesd -= Universal Binary Sections =- Architecture 0 (x86_64): CPU Type: 16777223 (0x1000007) CPU Subtype: 3 (0x3) CPU Subtype Capability: 0 (0x0) Offset: 0x4000 (Valid Mach-O Header: Yes) Size: 7922416 Align: 14 Architecture 1 (arm64e): CPU Type: 16777228 (0x100000c) CPU Subtype: 2 (0x2) CPU Subtype Capability: 128 (0x80) Offset: 0x794000 (Valid Mach-O Header: Yes) Size: 8783712 Align: 14 -= Found Symbol Offsets =- Offset of _IDSProtoKeyTransparencyTrustedServiceReadFrom in architecture x86_64: 0x0ccfdf Offset of _IDSProtoKeyTransparencyTrustedServiceReadFrom in architecture arm64e: 0x0b7570 -= Found Hex Offsets (with pure python fixed sequence search + regex) =- Architecture 0 (x86_64): IDSProtoKeyTransparencyTrustedServiceReadFrom: 0xccfdf NACInitAddress: 0x4ac060 NACKeyEstablishmentAddress: 0x48c0a0 NACSignAddress: 0x49f390 Architecture 1 (arm64e): IDSProtoKeyTransparencyTrustedServiceReadFrom: 0xb7570 NACInitAddress: 0x414e28 NACKeyEstablishmentAddress: 0x40268c NACSignAddress: 0x3dc898 ```

13.5

```bash ⇒ ./find_fat_binary_offsets.py samples/macos-13.5-22G74-ventura-x86-identityservicesd -= Universal Binary Sections =- Architecture 0 (x86_64): CPU Type: 16777223 (0x1000007) CPU Subtype: 3 (0x3) CPU Subtype Capability: 0 (0x0) Offset: 0x4000 (Valid Mach-O Header: Yes) Size: 7989040 Align: 14 Architecture 1 (arm64e): CPU Type: 16777228 (0x100000c) CPU Subtype: 2 (0x2) CPU Subtype Capability: 128 (0x80) Offset: 0x7a4000 (Valid Mach-O Header: Yes) Size: 8833808 Align: 14 -= Found Symbol Offsets =- Offset of _IDSProtoKeyTransparencyTrustedServiceReadFrom in architecture x86_64: 0x0cc743 Offset of _IDSProtoKeyTransparencyTrustedServiceReadFrom in architecture arm64e: 0x0b524c -= Found Hex Offsets (with pure python fixed sequence search + regex) =- Architecture 0 (x86_64): IDSProtoKeyTransparencyTrustedServiceReadFrom: 0xcc743 NACInitAddress: 0x4b91e0 NACKeyEstablishmentAddress: 0x499220 NACSignAddress: 0x4ac510 Architecture 1 (arm64e): IDSProtoKeyTransparencyTrustedServiceReadFrom: 0xb524c NACInitAddress: 0x41d714 NACKeyEstablishmentAddress: 0x40af78 NACSignAddress: 0x3e5184 ```

14.2

```bash ⇒ ./find_fat_binary_offsets.py samples/macos-14.2-sonoma-arm64-identityservicesd -= Universal Binary Sections =- Architecture 0 (x86_64): CPU Type: 16777223 (0x1000007) CPU Subtype: 3 (0x3) CPU Subtype Capability: 0 (0x0) Offset: 0x4000 (Valid Mach-O Header: Yes) Size: 8820512 Align: 14 Architecture 1 (arm64e): CPU Type: 16777228 (0x100000c) CPU Subtype: 2 (0x2) CPU Subtype Capability: 128 (0x80) Offset: 0x870000 (Valid Mach-O Header: Yes) Size: 9796976 Align: 14 -= Found Symbol Offsets =- Offset of _IDSProtoKeyTransparencyTrustedServiceReadFrom in architecture x86_64: 0x0d4899 Offset of _IDSProtoKeyTransparencyTrustedServiceReadFrom in architecture arm64e: 0x0bd9f0 -= Found Hex Offsets (with pure python fixed sequence search + regex) =- Architecture 0 (x86_64): IDSProtoKeyTransparencyTrustedServiceReadFrom: 0xd4899 NACInitAddress: 0x54c730 NACKeyEstablishmentAddress: 0x52c770 NACSignAddress: 0x53fa60 Architecture 1 (arm64e): IDSProtoKeyTransparencyTrustedServiceReadFrom: 0xbd9f0 NACInitAddress: 0x4b55a0 NACKeyEstablishmentAddress: 0x4a2e04 NACSignAddress: 0x47d010 ```

[rom4ster]

I have a mac on I believe 12.7.1 That I am willing to offer as a test case (I use this method and see if it works) if/when this gets implemented.

[0xdevalias]

@rom4ster Are you able to zip up the binary from the following path and upload it somewhere + share a link here as per:

Are you able to zip up the following file and upload it somewhere (ideally probably not this repo to avoid potential copyright/etc issues)?

/System/Library/PrivateFrameworks/IDS.framework/identityservicesd.app/Contents/MacOS/identityservicesd

Bonus points if you rename it to something like macos-14.1-beta-BUILDCODE-identityservicesd, where BUILDCODE is the bit of the version that looks similar to this 22E261

That way I (or someone) might be able to have a look and find the offsets for it (and can also test if my automatic script works on that version)

Originally posted by @0xdevalias in https://github.com/beeper/mac-registration-provider/issues/6#issuecomment-1868121844

[rom4ster]

@rom4ster Are you able to zip up the binary from the following path and upload it somewhere + share a link here as per:

Are you able to zip up the following file and upload it somewhere (ideally probably not this repo to avoid potential copyright/etc issues)?

/System/Library/PrivateFrameworks/IDS.framework/identityservicesd.app/Contents/MacOS/identityservicesd

Bonus points if you rename it to something like macos-14.1-beta-BUILDCODE-identityservicesd, where BUILDCODE is the bit of the version that looks similar to this 22E261 That way I (or someone) might be able to have a look and find the offsets for it (and can also test if my automatic script works on that version) Originally posted by @0xdevalias in #6 (comment)

Sure https://github.com/rom4ster/IdentityServices/blob/master/identityservicesd%20(1).txt

[0xdevalias]

Sure rom4ster/IdentityServices@master/identityservicesd%20(1).txt

@rom4ster Thanks for that :)

It looks like my current 'automatically find the offsets' code doesn't fully work on that version of identityservicesd from macOS Monterey 12.7.1:

⇒ sha256sum samples/macos-12.7.1-monterey-identityservicesd
5833338da6350266eda33f5501c5dfc793e0632b52883aa2389c438c02d03718  samples/macos-12.7.1-monterey-identityservicesd

⇒ ./find_fat_binary_offsets.py samples/macos-12.7.1-monterey-identityservicesd
-= Universal Binary Sections =-
Architecture 0 (x86_64):
  CPU Type: 16777223 (0x1000007)
  CPU Subtype: 3 (0x3)
  CPU Subtype Capability: 0 (0x0)
  Offset: 0x4000 (Valid Mach-O Header: Yes)
  Size: 7442432
  Align: 14
Architecture 1 (arm64e):
  CPU Type: 16777228 (0x100000c)
  CPU Subtype: 2 (0x2)
  CPU Subtype Capability: 128 (0x80)
  Offset: 0x720000 (Valid Mach-O Header: Yes)
  Size: 8707856
  Align: 14

-= Found Symbol Offsets =-
Offset of _IDSProtoKeyTransparencyTrustedServiceReadFrom in architecture x86_64: 0x0b2278
Offset of _IDSProtoKeyTransparencyTrustedServiceReadFrom in architecture arm64e: 0x0b562c

-= Found Hex Offsets (with pure python fixed sequence search + regex) =-
Architecture 0 (x86_64):
  IDSProtoKeyTransparencyTrustedServiceReadFrom: 0xb2278
  NACInitAddress: 0x4132e0
  NACKeyEstablishmentAddress:
  NACSignAddress:
Architecture 1 (arm64e):
  IDSProtoKeyTransparencyTrustedServiceReadFrom:
  NACInitAddress: 0x5897bc
  NACKeyEstablishmentAddress:
  NACSignAddress:

[rom4ster]

Do you know if the specified offsets exist? Like is it just that the symbols are in some other spot, or are the symbols themselves different on 12.7.1 ? I have not had the time to do a manual look with a decompiler (I tried with a hexeditor and it was... not so great).

If you do end up finding them manually or something, could you please update the provider or post them here (because might as well if you already did the work and also I kind of want to be able to use that mac.....)

[0xdevalias]

Do you know if the specified offsets exist? Like is it just that the symbols are in some other spot, or are the symbols themselves different on 12.7.1 ?

@rom4ster Manually reversed it (see below), and it looks like all the relevant functions still exist; so it's likely just an issue with the patterns my auto-tool is matching against.

---

(I tried with a hexeditor and it was... not so great)

@rom4ster Haha, I can only imagine! That sounds... painful :p

---

If you do end up finding them manually or something, could you please update the provider or post them here (because might as well if you already did the work and also I kind of want to be able to use that mac.....)

@rom4ster Yeah, absolutely. See below for the offsets I reversed this morning, and I will double check them a bit later + open a PR to have them added to this tool. They should be correct, but I want to be triple sure before I submit a PR with them.

---

It looks like my current 'automatically find the offsets' code doesn't fully work on that version of identityservicesd from macOS Monterey 12.7.1:

Manually reversing the offsets (assuming I didn't make a mistake):

samples/macos-12.7.1-monterey-identityservicesd

sha256: 5833338da6350266eda33f5501c5dfc793e0632b52883aa2389c438c02d03718

x86

IDSProtoKeyTransparencyTrustedServiceReadFrom: 0xb2278 (0x1000b2278) nac_init: 0x4132e0 (0x1004132e0) nac_key_establishment: 0x465e00 (0x100465e00) nac_sign: 0x405c10 (0x100405c10)

arm64

IDSProtoKeyTransparencyTrustedServiceReadFrom: 0x0b562c (0x1000b562c) nac_init: 0x43d408 (0x10043d408) (??? auto tool found this, but don't think it's correct.. 0x5897bc (0x1005897bc) ???) nac_key_establishment: 0x3fdafc (0x1003fdafc) nac_sign: 0x3f2844 (0x1003f2844)

Edit: Opened a PR to add these offsets:

• https://github.com/beeper/mac-registration-provider/pull/18

---

I'll see if I can use this to refine the patterns used in find_fat_binary_offsets.py

[0xdevalias]

@jetfir3 I just saw your PR to add a bunch of other offsets (https://github.com/beeper/mac-registration-provider/pull/12), and was wondering if you'd be able to upload the identityservicesd binaries for those versions somewhere as per this comment so I can test/refine my auto-offset-finding code.

[jetfir3]

Here's a zip: https://oshi.at/vPMG (expires in ~72hrs)

Since these are not U2B, no 0x4000 padding calculation will be needed.

On 10.14.x and earlier, IDSProtoKeyTransparencyTrustedServiceReadFrom is not present, so I chose another symbol to use for the 10.14.6 offsets I provided -- _newLocalDeliveryServiceStatString. There wasn't any specific reasoning behind that selection and I suppose any symbol would do.

The patterns below will find all 4 offsets for all of 10.15.x:

ReferenceAddress: "\x55\x48\x89\xE5\x41\x57\x41\x56\x41\x55\x41\x54\x53\x48\x83\xEC\x28\x49\x89\xF6\x48\x89\x7D\xD0\x4C\x8B\x2D..\x4D\x00\x4D"
NACInitAddress: "\x55\x48\x89\xE5\x41\x57\x41\x56\x41\x55\x41\x54\x53\x48\x81\xEC\x48\x18\x00\x00\x49\x89\xD6\x48\x8B\x05..\x1C\x00\x48\x8B"
NACKeyEstablishmentAddress: "\x55\x48\x89\xE5\x41\x57\x41\x56\x41\x55\x41\x54\x53\x48\x81\xEC\x28\x01\x00\x00\x48\x8B\x05..\x1B\x00\x48\x8B\x00\x48\x89\x45\xD0\x48\x85"
NACSignAddress: "\x55\x48\x89\xE5\x41\x57\x41\x56\x41\x55\x41\x54\x53\x48\x81\xEC\x58\x03\x00\x00\x41\x89\xD1\x49\x89\xFF\x48\x8B\x05..\x1C"

[0xdevalias]

The following issue is about 11.0.1; hopefully they upload the binary so it can be reversed:

• https://github.com/beeper/mac-registration-provider/issues/10

[0xdevalias]

Here's a zip: LINK (expires in ~72hrs)

The patterns below will find all 4 offsets for all of 10.15.x:

@jetfir3 Awesome, thanks for that! :)

[0xdevalias]

Added a few more helper scripts + tool/pattern refinements + hex dumps + etc:

• https://github.com/0xdevalias/poc-re-binsearch/commits/main/?since=2023-12-30&until=2023-12-31
  • Notably: diff_hex.py arg improvements, hex-dump-helper

For the pattern refinements in those commits, I didn't use the hex dumps/offsets from the 10.x patterns as ./find_fat_binary_offsets.py needs to be refined to work with non-universal binaries:

# TODO: Need to refactor this script to handle non-fat binaries, and then we can try the following patterns:
#   ⇒ ./find_fat_binary_offsets.py --shortest-patterns samples/macos-10.14.6-mojave-x86_64-identityservicesd
#   -= Universal Binary Sections =-
#   Traceback (most recent call last):
#     File "/Users/devalias/dev/0xdevalias/poc-re-binsearch/./find_fat_binary_offsets.py", line 703, in <module>
#       main()
#     File "/Users/devalias/dev/0xdevalias/poc-re-binsearch/./find_fat_binary_offsets.py", line 120, in main
#       print_arch_info(architectures)
#     File "/Users/devalias/dev/0xdevalias/poc-re-binsearch/./find_fat_binary_offsets.py", line 289, in print_arch_info
#       for i, arch in architectures.items():
#   AttributeError: 'str' object has no attribute 'items'
## Patterns from 10.x binaries
# hex_strings = {
#     "x86_64": {
#         "IDSProtoKeyTransparencyTrustedServiceReadFrom": "554889e54157415641554154534883ec284989f648897dd04c8b2d....4d004d8b45004a8b0c064c8b3d....4d00498b17483b0c160f83..010000488b05....5700488945c0488b05....5700488945c84c8b25....4d004c8b0d....4d00498b0c2441",
#         "NACInitAddress": "554889e54157415641554154534881ec481800004989d6488b05......00488b00488945d00f314889d348c1e3204809c348ba8fe3388ee3388ee34889d848f7e248c1ea03488d04d24829c34889d848c1e00448ba7febeb767bf7f7fe4831c24883e307",
#         "NACKeyEstablishmentAddress": "554889e54157415641554154534881ec28010000488b05......00488b00488945d04885ff0f94c04885f60f94c108c185d20f94c008c84189d70fb6c08d48064c8d25......0049630c8c488d15e5ffffff4801ca41bd05514f51ffe24656904889f248",
#         "NACSignAddress": "554889e54157415641554154534881ec580300004189d14989ff488b05......00488b00488945d0488d95d0fcffff488995d8fcffff89d02548488a9489d381e3109524214189d24181e2a022514a8d3c1289fa81e20000042809c281f20200029489f8",
#     },
#     "arm64e": {
#         "IDSProtoKeyTransparencyTrustedServiceReadFrom": "N/A",
#         "NACInitAddress": "N/A",
#         "NACKeyEstablishmentAddress": "N/A",
#         "NACSignAddress": "N/A",
#     },
# }

The updated patterns still don't uniquely match for everything, namely, for the 12.7.1/13.3.1/13.5/14.2 binaries, see below:

• NACSignAddress (x86_64 on 12.7.1)
• IDSProtoKeyTransparencyTrustedServiceReadFrom (arm64e on 12.7.1, 13.3.1, 13.5, 14.2)
  • Though this can be read in a normal 'read symbols' way, so it's not super important to have a unique pattern for it

[0xdevalias]

Opened a PR to add the 12.7.1 offsets from above (Ref):

• https://github.com/beeper/mac-registration-provider/pull/18

[0xdevalias]

Here is the binary for 12.7.2:

⇒ sha256sum macos-12.7.2-monterey-identityservicesd
01aaa511c5d32c5766256a40b5ae8f42fb49b74074dce5936f315244236f15a0  macos-12.7.2-monterey-identityservicesd

Using my automated script (see https://github.com/beeper/mac-registration-provider/issues/9), this is the output it gave:

⇒ ./find_fat_binary_offsets.py samples/macos-12.7.2-monterey-identityservicesd
-= Universal Binary Sections =-
Architecture 0 (x86_64):
  CPU Type: 16777223 (0x1000007)
  CPU Subtype: 3 (0x3)
  CPU Subtype Capability: 0 (0x0)
  Offset: 0x4000 (Valid Mach-O Header: Yes)
  Size: 7442432
  Align: 14
Architecture 1 (arm64e):
  CPU Type: 16777228 (0x100000c)
  CPU Subtype: 2 (0x2)
  CPU Subtype Capability: 128 (0x80)
  Offset: 0x720000 (Valid Mach-O Header: Yes)
  Size: 8707856
  Align: 14

-= Found Symbol Offsets =-
Offset of _IDSProtoKeyTransparencyTrustedServiceReadFrom in architecture x86_64: 0x0b2278
Offset of _IDSProtoKeyTransparencyTrustedServiceReadFrom in architecture arm64e: 0x0b562c

-= Found Hex Offsets (with pure python fixed sequence search + regex) =-
Architecture 0 (x86_64):
  IDSProtoKeyTransparencyTrustedServiceReadFrom: 0xb2278
  NACInitAddress: 0x4132e0
  NACKeyEstablishmentAddress: 0x465e00
  NACSignAddress: 0x103f47; 0x405c10
Architecture 1 (arm64e):
  IDSProtoKeyTransparencyTrustedServiceReadFrom: 0xb562c; 0x31a038; 0x33203c
  NACInitAddress: 0x43d408
  NACKeyEstablishmentAddress: 0x3fdafc
  NACSignAddress: 0x3f2844

Note that there are multiple offsets listed for some parts, so those would need to be narrowed down further via manual/other methods to get the full proper offsets needed.

Originally posted by @0xdevalias in https://github.com/beeper/mac-registration-provider/issues/23#issuecomment-1902814913

[0xdevalias]

identityservicesd 14.3 (23D56)

⇒ sha256sum macos-14.3-23D56-sonoma-identityservicesd
d3c6986fefcbd2efea2a8a7c88104bf22d60d1f4f2bbf3615a1e3ce098aba765  macos-14.3-23D56-sonoma-identityservicesd

Using my automated script (see https://github.com/beeper/mac-registration-provider/issues/9), this is the output it gave:

⇒ ./find_fat_binary_offsets.py samples/macos-14.3-23D56-sonoma-identityservicesd
-= Universal Binary Sections =-
Architecture 0 (x86_64):
  CPU Type: 16777223 (0x1000007)
  CPU Subtype: 3 (0x3)
  CPU Subtype Capability: 0 (0x0)
  Offset: 0x4000 (Valid Mach-O Header: Yes)
  Size: 8820512
  Align: 14
Architecture 1 (arm64e):
  CPU Type: 16777228 (0x100000c)
  CPU Subtype: 2 (0x2)
  CPU Subtype Capability: 128 (0x80)
  Offset: 0x870000 (Valid Mach-O Header: Yes)
  Size: 9796976
  Align: 14

-= Found Symbol Offsets =-
Offset of _IDSProtoKeyTransparencyTrustedServiceReadFrom in architecture x86_64: 0x0d47c9
Offset of _IDSProtoKeyTransparencyTrustedServiceReadFrom in architecture arm64e: 0x0bd81c

-= Found Hex Offsets (with pure python fixed sequence search + regex) =-
Architecture 0 (x86_64):
  IDSProtoKeyTransparencyTrustedServiceReadFrom: 0xd47c9
  NACInitAddress: 0x54c6d0
  NACKeyEstablishmentAddress: 0x52c710
  NACSignAddress: 0x53fa00
Architecture 1 (arm64e):
  IDSProtoKeyTransparencyTrustedServiceReadFrom: 0xbd81c; 0x2ef2d0; 0x31c2d4; 0x333754
  NACInitAddress: 0x4b5580
  NACKeyEstablishmentAddress: 0x4a2de4
  NACSignAddress: 0x47cff0

Given the only part with multiple offsets is the IDSProtoKeyTransparencyTrustedServiceReadFrom symbol (that we have the proper offset for by directly looking it up), I assume these should be fine; though I haven't manually verified them at all.

Edit: Actually, these offsets seem to match with offsets_14_3_b3, though the hash is different:

https://github.com/beeper/mac-registration-provider/blob/673616637da32adea83e316cc98980b6caebc178/nac/offsets.go#L219-L234

Originally posted by @0xdevalias in https://github.com/beeper/mac-registration-provider/issues/27#issuecomment-1902813111

[TheDave94]

I have extracted the offsets for macos 14.4 beta2. Would it be possible to add them so I can create a new registration code?

-= Universal Binary Sections =- Architecture 0 (x86_64): CPU Type: 16777223 (0x1000007) CPU Subtype: 3 (0x3) CPU Subtype Capability: 0 (0x0) Offset: 0x4000 (Valid Mach-O Header: Yes) Size: 8866912 Align: 14 Architecture 1 (arm64e): CPU Type: 16777228 (0x100000c) CPU Subtype: 2 (0x2) CPU Subtype Capability: 128 (0x80) Offset: 0x87c000 (Valid Mach-O Header: Yes) Size: 9847584 Align: 14

-= Found Symbol Offsets =- Offset of _IDSProtoKeyTransparencyTrustedServiceReadFrom in architecture x86_64: 0x0d5a35 Offset of _IDSProtoKeyTransparencyTrustedServiceReadFrom in architecture arm64e: 0x0bec84

-= Found Hex Offsets (with pure python fixed sequence search + regex) =- Architecture 0 (x86_64): IDSProtoKeyTransparencyTrustedServiceReadFrom: 0xd5a35 NACInitAddress: 0x5558a0 NACKeyEstablishmentAddress: 0x5358e0 NACSignAddress: 0x548bd0 Architecture 1 (arm64e): IDSProtoKeyTransparencyTrustedServiceReadFrom: 0xbec84; 0x2f33c4; 0x320464; 0x3378cc NACInitAddress: 0x4bf1d8 NACKeyEstablishmentAddress: 0x4aca3c NACSignAddress: 0x486c48

[0xdevalias]

@TheDave94 You could open a pull request to add them following the similar examples in:

• https://github.com/beeper/mac-registration-provider/blob/main/nac/offsets.go

See also:

• https://github.com/beeper/mac-registration-provider/issues/34
  • https://github.com/beeper/mac-registration-provider/pull/36

[Marckus]

I tried it on my 2017 MacBook Pro running Ventura 13.6.4 and received "No offsets found for 13.6.4/22G513/amd64". From what I've gleaned from this site, it should have worked, no? I ran the python magic script and this was its output, which appears to be in the offsets file:

-= Universal Binary Sections =- Architecture 0 (x86_64): CPU Type: 16777223 (0x1000007) CPU Subtype: 3 (0x3) CPU Subtype Capability: 0 (0x0) Offset: 0x4000 (Valid Mach-O Header: Yes) Size: 7989040 Align: 14 Architecture 1 (arm64e): CPU Type: 16777228 (0x100000c) CPU Subtype: 2 (0x2) CPU Subtype Capability: 128 (0x80) Offset: 0x7a4000 (Valid Mach-O Header: Yes) Size: 8833808 Align: 14

-= Found Symbol Offsets =- Offset of _IDSProtoKeyTransparencyTrustedServiceReadFrom in architecture x86_64: 0x0cc743 Offset of _IDSProtoKeyTransparencyTrustedServiceReadFrom in architecture arm64e: 0x0b524c

-= Found Hex Offsets (with pure python fixed sequence search + regex) =- Architecture 0 (x86_64): IDSProtoKeyTransparencyTrustedServiceReadFrom: 0xcc743 NACInitAddress: 0x4b91e0 NACKeyEstablishmentAddress: 0x499220 NACSignAddress: 0x4ac510 Architecture 1 (arm64e): IDSProtoKeyTransparencyTrustedServiceReadFrom: 0xb524c; 0x2d4a14; 0x300a04; 0x316db4 NACInitAddress: 0x41d714 NACKeyEstablishmentAddress: 0x40af78 NACSignAddress: 0x3e5184

[0xdevalias]

@Marckus That only looks like partial output from my python script (unless it just crashed/exited at that point?

Also, what's the sha256sum of the identity service binary? (Which I believe also is included in the full error message)

You can then check if that hash is in the offsets file in the code in this repository; that'll confirm whether your version is supported or not.

[Marckus]

OK, I'll try that to check. Also, I updated my post as it did crash, I was using the wrong binary.

[Marckus]

@Marckus That only looks like partial output from my python script (unless it just crashed/exited at that point?

Also, what's the sha256sum of the identity service binary? (Which I believe also is included in the full error message)

You can then check if that hash is in the offsets file in the code in this repository; that'll confirm whether your version is supported or not.

It IS in the offsets file, so this is even more puzzling...

2024/03/05 20:53:03 Starting mac-registration-provider 31dc297a 2024/03/05 20:53:03 Loading identityservicesd 2024/03/05 20:53:03 No offsets found for 13.6.4/22G513/amd64 (hash: 8f22dcfda56a4d3c38931f20fe33db1a6720e4d8571e452aa5a8b56b4c69842a)

[0xdevalias]

@Marckus Next thing to check is whether the version of this binary you're running is actually the latest one; or at least, one that includes those offsets. Often in situations like this it's because the user is running an old version of this tool.

31dc297 is the commit of the version of the tool you're running; if the offsets file at that code revision doesn't have it; that would be your problem.

[Marckus]

Thanks, that was it! A lot of good it'll do me now though, Beeper has apparently pulled iMessage altogether now. I can't even select/add it as a Chat Network any more. :(

[Marckus]

Ventura 13.6.5 is out now and I forgot about Beeper and upgraded. If someone wants to put the new lines into offsets.go and rebuild, that would be awesome:

// macOS 13.6.5 hexToByte32("253e81362b42a1a5db4270a0f7b55f3f7f0b0ed4c8e3f8a45660d22e3025a971"): offsets_13_6,

[0xdevalias]

You can see more of my methodology for finding the offsets using this tool in the following comments:

• https://github.com/beeper/mac-registration-provider/issues/47#issuecomment-2054451356
• https://github.com/beeper/mac-registration-provider/issues/47#issuecomment-2054839669

---

Looking at a bunch of the existing 10.14.x/10.15.x offsets, we can observe that:

• NACKeyEstablishmentAddress - NACSignAddress = 0x14660
• NACKeyEstablishmentAddress - NACInitAddress = 0xebc0

If we assume that same pattern may hold on this binary, we can attempt to narrow down the likely matching offset:

• NACSignAddress: NACKeyEstablishmentAddress (0x3478a0) - 0x14660 = 0x333240
  • This appears to be one of the offsets listed above for NACSignAddress, so may be what we want.
• NACInitAddress: NACKeyEstablishmentAddress (0x3478a0) - 0xebc0 = 0x338ce0
  • This appears to be one of the offsets listed above for NACInitAddress, so may be what we want.

Originally posted by @0xdevalias in https://github.com/beeper/mac-registration-provider/issues/47#issuecomment-2054839669

---

And these patterns seem to work for both of the 10.14.6 binaries, to find the unique offsets automagically:

symbol = "_newLocalDeliveryServiceStatString" # For 10.14.x and earlier

# 10.14.6 unique match (non-minified patterns)
# hex_strings = {
#     "x86_64": {
#         f"ReferenceAddress ({symbol})": "554889e54157415641554154534881ec98010000488b05....2800488b00488945d0ff15....28004889c348898530ffffff488b3d....31004c8b35....3100",
#         "NACInitAddress": "554889e54157415641554154534881ec481800004989d6488b05......00488b00488945d00f314889d348c1e3204809c348ba8fe3388ee3388ee34889d848f7e248c1ea03488d04d24829c34889d848c1e00448ba7febeb767bf7f7fe4831c24883e307",
#         "NACKeyEstablishmentAddress": "554889e54157415641554154534881ec28010000488b05......00488b00488945d04885ff0f94c04885f60f94c108c185d20f94c008c84189d70fb6c08d48064c8d25......0049630c8c488d15e5ffffff4801ca41bd05514f51ffe24656904889f248",
#         "NACSignAddress": "554889e54157415641554154534881ec580300004189d14989ff488b05......00488b00488945d0488d95d0fcffff488995d8fcffff89d02548488a9489d381e3109524214189d24181e2a022514a8d3c1289fa81e20000042809c281f20200029489f8",
#     }
# }

# 10.14.6 unique match (minified patterns)
hex_strings = {
    "x86_64": {
        f"ReferenceAddress ({symbol})": "554889e54157415641554154534881ec98010000488b05....28",
        "NACInitAddress": "554889e54157415641554154534881ec4818",
        "NACKeyEstablishmentAddress": "554889e54157415641554154534881ec28010000488b05......00488b00488945d04885",
        "NACSignAddress": "554889e54157415641554154534881ec5803",
    }
}

⇒ ./find_fat_binary_offsets.py samples/macos-10.14.6-mojave-x86_64-identityservicesd
-= Universal Binary Sections =-
Architecture 0 (x86_64):
  CPU Type: 0 (0x0)
  CPU Subtype: 0 (0x0)
  CPU Subtype Capability: 0 (0x0)
  Offset: 0x0 (Valid Mach-O Header: Yes)
  Size: -1
  Align: 0

-= Found Symbol Offsets =-
Offset of _newLocalDeliveryServiceStatString in architecture x86_64: 0x238842

-= Found Hex Offsets (with pure python fixed sequence search + regex) =-
Architecture 0 (x86_64):
  ReferenceAddress (_newLocalDeliveryServiceStatString): 0x238842
  NACInitAddress: 0x338ce0
  NACKeyEstablishmentAddress: 0x3478a0
  NACSignAddress: 0x333240

⇒ ./find_fat_binary_offsets.py samples/macos-10.14.6-18G103-mojave-x86_64-identityservicesd
-= Universal Binary Sections =-
Architecture 0 (x86_64):
  CPU Type: 0 (0x0)
  CPU Subtype: 0 (0x0)
  CPU Subtype Capability: 0 (0x0)
  Offset: 0x0 (Valid Mach-O Header: Yes)
  Size: -1
  Align: 0

-= Found Symbol Offsets =-
Offset of _newLocalDeliveryServiceStatString in architecture x86_64: 0x238c26

-= Found Hex Offsets (with pure python fixed sequence search + regex) =-
Architecture 0 (x86_64):
  ReferenceAddress (_newLocalDeliveryServiceStatString): 0x238c26
  NACInitAddress: 0x338ce0
  NACKeyEstablishmentAddress: 0x3478a0
  NACSignAddress: 0x333240

Originally posted by @0xdevalias in https://github.com/beeper/mac-registration-provider/issues/47#issuecomment-2054978827

[0xdevalias]

For reference, these are all of the patterns I have in my local copy of the auto-finder tool now:

# Symbol to search for
# symbol = "_newLocalDeliveryServiceStatString" # For 10.14.x and earlier
symbol = "_IDSProtoKeyTransparencyTrustedServiceReadFrom" # For 10.15.x onwards

# 10.14.x (and earlier?)
# See also:
#   https://github.com/beeper/mac-registration-provider/issues/47#issuecomment-2054451356
#   https://github.com/beeper/mac-registration-provider/issues/47#issuecomment-2054839669
#     Looking at a bunch of the existing 10.14.x/10.15.x offsets, we can observe that:
#
#     - NACKeyEstablishmentAddress - NACSignAddress = 0x14660
#     - NACKeyEstablishmentAddress - NACInitAddress = 0xebc0
#
#     If we assume that same pattern may hold on this binary, we can attempt to narrow down the likely matching offset:
#
#     - NACSignAddress: NACKeyEstablishmentAddress (0x3478a0) - 0x14660 = 0x333240
#       - This appears to be one of the offsets listed above for NACSignAddress, so may be what we want.
#     - NACInitAddress: NACKeyEstablishmentAddress (0x3478a0) - 0xebc0 = 0x338ce0
#       - This appears to be one of the offsets listed above for NACInitAddress, so may be what we want.
# 10.14.x unique match (mostly minified patterns (that still work with 10.15.x))
hex_strings_10_14_x = {
    "x86_64": {
        f"ReferenceAddress ({symbol})": "554889e54157415641554154534881ec98010000488b05....28",
        "NACInitAddress": "554889e54157415641554154534881ec4818",
        "NACKeyEstablishmentAddress": "554889e54157415641554154534881ec28010000488b05......00488b00488945d04885",
        "NACSignAddress": "554889..4157415641554154534881..580300004189..4989..48..05....",
    }
}

# 10.15.x unique match (minified patterns)
hex_strings_10_15_x = {
    "x86_64": {
        f"ReferenceAddress ({symbol})": "554889e54157415641554154534883ec284989f648897dd04c8b2d",
        "NACInitAddress":               hex_strings_10_14_x['x86_64']['NACInitAddress'],
        "NACKeyEstablishmentAddress":   hex_strings_10_14_x['x86_64']['NACKeyEstablishmentAddress'],
        "NACSignAddress":               hex_strings_10_14_x['x86_64']['NACSignAddress'],
    }
}

# Current main one..
hex_strings_modern = {
    "x86_64": {
        f"ReferenceAddress ({symbol})": "554889e54157415641554154534883ec28..89..48897dd04c8b3d",
        "NACInitAddress": "554889e541574156415541545350b87818",
        "NACKeyEstablishmentAddress": "554889e54157415641554154534881ec..010000488b05......00488b00488945d04885",
        "NACSignAddress": "554889e54157415641554154534881ec..030000........................................................................................................................................................................................48....48..........................................................................................................89............................................................",  # TODO: Find a pattern that gets unique matches for this
    },
    "arm64e": {
        f"ReferenceAddress ({symbol})": "7f2303d5ffc301d1fc6f01a9fa6702a9f85f03a9f65704a9f44f05a9fd7b06a9fd830191f30301aa....00........f9..0280b9..68..f8....00........f9....80b9..68..f8....00........f9..01..eb....0054f40300aa............................................................................................................................80b96d6a6df89f010deb....0054..0380b96d6a6df8................................................",  # TODO: Find a pattern that gets unique matches for this
        "NACInitAddress": "7f2303d5fc6fbaa9fa6701a9f85f02a9f65703a9f44f04a9fd7b05a9fd43019109..8352....00..10....f91f0a3fd6ff0740d1ff....d1....00..08....f9080140f9a8....f8......d2......f2......f2......f2e9",
        "NACKeyEstablishmentAddress": "7f2303d5ff....d1fc6f..a9fa67..a9f85f..a9f657..a9f44f..a9fd7b..a9fd..0591....00..08....f9080140f9a8....f8......52",
        "NACSignAddress": "7f2303d5fc6fbaa9fa6701a9f85f02a9f65703a9f44f04a9fd7b05a9fd430191ff....d1................08....f9......................................................................................................................................f2......f2......................d2",
    },
}

# hex_strings = hex_strings_10_14_x
# hex_strings = hex_strings_10_15_x
hex_strings = hex_strings_modern

Edit: Have also pushed these to my repo in https://github.com/0xdevalias/poc-re-binsearch/commit/573c1259fe6967e870130726c47ecb657bed838b

[0xdevalias]

There is some discussion about finding the offsets for macOS 15 binaries here:

• https://github.com/0xdevalias/poc-re-binsearch/issues/2

Including some of my notes on how to manually reverse the offsets in this comment:

• https://github.com/0xdevalias/poc-re-binsearch/issues/2#issuecomment-2204900396