binary signed with ldid cannot have its entitlements read by `codesign`

[asdfzxcvbn]

this issue was discovered by @teflocarbon here

you can read a summary of the issue here: https://github.com/asdfzxcvbn/pyzule-rw/issues/8#issuecomment-2439338344

but the important part is that signtools reads entitlements using: run_process("codesign", "--no-strict", "-d", "--entitlements", ":-", str(component))

this process fails and codesign prints the following to stderr:

Executable=/private/var/folders/d3/3v3wd4390c76s9j518cn498c0000gn/T/tmpcz06ndqj/Payload/Discord.app/Discord
/var/folders/d3/3v3wd4390c76s9j518cn498c0000gn/T/tmpcz06ndqj/Payload/Discord.app: no signature
warning: Specifying ':' in the path is deprecated and will not work in a future release
warning: binary contains an invalid entitlements blob. The OS will ignore these entitlements.

it should be noted that the issue is fixed by patching signtools CI to use ldid to read the entitlements of codesign, but codesign should be able to read the entitlements of a binary signed with ldid

the author also stated that the issue is apparently fixed when the entitlements plist is exported with ldid and then the binary is signed with codesign

so i believe this is likely an ldid error

[CRKatri]

So to be clear, codesign is unable to dump entitlements from one specific ldid signed binary? Can you reproduce this with:

1. A smaller binary (not a .app, just a standalone Mach-O)
2. With a smaller entitlement list

What macOS, Xcode, and codesign binary versions are you using?

[asdfzxcvbn]

So to be clear, codesign is unable to dump entitlements from one specific ldid signed binary?

exactly

Can you reproduce this with:

1. A smaller binary (not a .app, just a standalone Mach-O)

2. With a smaller entitlement list

What macOS, Xcode, and codesign binary versions are you using?

paraphrasing from @whoeevee now:

it's reproducable with a single binary. even after only leaving two entitlements (aps-environment and application-identifier), it still causes an error with codesign.

$ codesign --no-strict -d --entitlements :- TikTok
Executable=/Users/eevee/Downloads/tt/Payload 3/TikTok.app/TikTok
warning: Specifying ':' in the path is deprecated and will not work in a future release
<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "https://www.apple.com/DTDs/PropertyList-1.0.dtd"><plist version="1.0"><dict><key>application-identifier</key><string>MJ797D8U6F.com.zhiliaoapp.musically</string>

$ ldid -e TikTok > ent
$ ldid -Sent TikTok

$ codesign --no-strict -d --entitlements :- TikTok
Executable=/Users/eevee/Downloads/tt/Payload 3/TikTok.app/TikTok
TikTok: no signature
warning: Specifying ':' in the path is deprecated and will not work in a future release
warning: binary contains an invalid entitlements blob. The OS will ignore these entitlements.

this is with macOS v14.7 and xcode v16.0, all on stable releases, and apparently the issue is reproducable on all system versions. also should note that codesign -dvvv reports no signature after an ldid -Sent:

$ codesign -dvvv TikTok
Executable=/Users/eevee/Downloads/tt/Payload 3/TikTok.app/TikTok
Identifier=TikTok
Format=app bundle with Mach-O thin (arm64)
CodeDirectory v=20400 size=927 flags=0x0(none) hashes=19+7 location=embedded
Hash type=sha256 size=32
CandidateCDHash sha1=46d249b8cb849859c8a267f3399c516e0b67fb86
CandidateCDHashFull sha1=46d249b8cb849859c8a267f3399c516e0b67fb86
CandidateCDHash sha256=ed57a6c57b2a15b2d4ed55991d87f3c0dad5dc9b
CandidateCDHashFull sha256=ed57a6c57b2a15b2d4ed55991d87f3c0dad5dc9b244975cd0011afbd3d936c4f
Hash choices=sha1,sha256
CMSDigest=a3dd1f7984cf149b8e3dec12339919fd72c412a35fb2e4f75269f0d8313648d4
CMSDigestType=2
CDHash=ed57a6c57b2a15b2d4ed55991d87f3c0dad5dc9b
TikTok: no signature <---
Info.plist=not bound <---
TeamIdentifier=not set <---
Sealed Resources=none
Internal requirements count=1 size=124

[teflocarbon]

macOS Version: 15.1 Beta (24B5077a) Xcode Version: 16.0 (16A242d) Xcode CLI Tools Version: 16.0.0.0.1.1724870825

To chime in here, I used a smaller entitlement list with only one key and still experienced the same behaviour. I also did it with a smaller non .app binary as requested.

I used this one here. https://github.com/JonathanSalwan/binary-samples/blob/master/MachO-OSX-x64-ls

These was the entitlements I tested with.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>com.apple.security.app-sandbox</key>
    <true/>
</dict>
</plist>

I couldn't find the codesign version but it comes with Xcode CLI Tools so I provided the CLI tools version.

[CRKatri]

Can I get the version of ldid, and the libplist and openssl it was built against? Are you using your own build or a build artifact from our CI?

[asdfzxcvbn]

Can I get the version of ldid, and the libplist and openssl it was built against? Are you using your own build or a build artifact from our CI?

we're using the build artifacts from the latest release

[CRKatri]

Could you try from the latest commit? Idk why there are no artifacts so you’ll have to build it yourself, but you can just follow the steps in the workflow file, it’s pretty easy.

[asdfzxcvbn]

Could you try from the latest commit? Idk why there are no artifacts so you’ll have to build it yourself, but you can just follow the steps in the workflow file, it’s pretty easy.

the issue still persists with the latest commit. i built it on my own pc, with chaotic-aur/libplist-git 1:2.6.0.r2.ge8791e2-1 and core/openssl 3.3.2-1

however i actually just discovered something weird, it works with the codesign bundled in xcode v13.2.1 (binaries from both old and new ldid commits)

[CRKatri]

Hmm, could you sign a binary with the same entitlements with both codesign and ldid, and attach them, so that I can compare the generated code signature? I’ll diff them and figure out what we’re generating wrong.

[asdfzxcvbn]

sure, here's the example Mach-O generated with ldid -Sent MachO-ldid: https://files.catbox.moe/byuxqs

and the binary generated with codesign --entitlements ent -f -s - MachO-codesign: https://files.catbox.moe/lh8lun

perhaps something interesting to note is the size difference (34.8kb with ldid vs 52.3kb with codesign)

[CRKatri]

So I'm looking at the signatures in both of these binaries and the xml entitlements and DER entitlements are identical. The main difference I'm seeing is that the ldid signed binary doesn't have adhoc flag, and has some requirements, and the codesign signed binary is adhoc, has empty requirements, and has an empty signature slot.

[asdfzxcvbn]

So I'm looking at the signatures in both of these binaries and the xml entitlements and DER entitlements are identical. The main difference I'm seeing is that the ldid signed binary doesn't have adhoc flag, and has some requirements, and the codesign signed binary is adhoc, has empty requirements, and has an empty signature slot.

is ldid expected to be doing the same things as codesign? or will ldid be changed to match codesign behavior

[CRKatri]

Well the adhoc flag is missing cause you didn’t sign with -Cadhoc. Idk why codesign decided to add an empty Signature slot, that seems like different behavior than older versions cause I don’t remember that happening in the past. The requirements thing is something that ldid does by default and I’d have to do research on whether that is good behavior or if it should be changed.

[teflocarbon]

Using -Cadhoc allowed it to be viewed by both of them. Seems like older Xcode versions didn't require it, which is interesting.

Without using -Cadhoc it fails to read with codesign.

./ldid -SEntitlements-test.plist MachO-OSX-x64-ls

./ldid -e MachO-OSX-x64-ls
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>com.apple.security.app-sandbox</key>
    <true/>
</dict>
</plist>

codesign -d --entitlements :- MachO-OSX-x64-ls --no-strict
Executable=/Users/teflocarbon/Downloads/MachO-OSX-x64-ls
MachO-OSX-x64-ls: no signature
warning: Specifying ':' in the path is deprecated and will not work in a future release
warning: binary contains an invalid entitlements blob. The OS will ignore these entitlements.

When using -Cadhoc it resolves the issue.

./ldid -SEntitlements-test.plist -Cadhoc MachO-OSX-x64-ls

./ldid -e MachO-OSX-x64-ls
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>com.apple.security.app-sandbox</key>
    <true/>
</dict>
</plist>

codesign -d --entitlements :- MachO-OSX-x64-ls --no-strict
Executable=/Users/teflocarbon/Downloads/MachO-OSX-x64-ls
warning: Specifying ':' in the path is deprecated and will not work in a future release
<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "https://www.apple.com/DTDs/PropertyList-1.0.dtd"><plist version="1.0"><dict><key>com.apple.security.app-sandbox</key><true/></dict></plist>

[CRKatri]

Very interesting, is that resolution enough for you? What happens if you have no adhoc flag, but legit sign something?

[asdfzxcvbn]

What happens if you have no adhoc flag, but legit sign something?

for our usecase, we don't need to actually sign it legitimately, we just needed an adhoc signature. this is resolved for us but you may want to keep the issue open for that signature slot/requirements thing? thanks for your help!