search

iterate-ch / cyberduck

Cyberduck is a libre FTP, SFTP, WebDAV, Amazon S3, Backblaze B2, Microsoft Azure & OneDrive and OpenStack Swift file transfer client for Mac and Windows.
https://cyberduck.io/
GNU General Public License v3.0
3.22k stars 286 forks source link

Receipt validation from Mac App Store fails on macOS 15 beta #16031

Closed Bettarg closed 4 days ago

Bettarg commented 2 months ago

This ticket system is to report bugs and feature requests. For support, visit the help page first.

Very simple , the App crash on Startup. In Detail, MacOS claimed the App is corrupted. Reinstall does not help.

dkocher commented 2 months ago

I cannot reproduce on 15.0 beta, 24A5264n.

matthewberryman commented 2 months ago

To reproduce, you need the version from the App Store:

Screenshot 2024-06-20 at 10 25 55

The version from outside the App Store doesn't have this issue.

Edit: it's to do with MAC address rotation, see my later comment

matthewberryman commented 2 months ago

Looking through the logs:

2024-06-17 07:40:31,446 [main] ERROR ch.cyberduck.core.aquaticprime.ReceiptVerifier - Failed verification. Hash with GUID redacted_hex_string does not match hash in receipt
2024-06-17 14:00:48,063 [main] ERROR ch.cyberduck.core.aquaticprime.ReceiptVerifier - Failed verification. Hash with GUID different_redacted_hex_string not match hash in receipt
matthewberryman commented 2 months ago 
matthew@MacBookPro /A/C/C/MacOS> codesign -vvv --deep --strict ./Cyberduck
--prepared:/Applications/Cyberduck.app/Contents/PlugIns/Runtime.jre
--validated:/Applications/Cyberduck.app/Contents/PlugIns/Runtime.jre
--prepared:/Applications/Cyberduck.app/Contents/Frameworks/librococoa.dylib
--validated:/Applications/Cyberduck.app/Contents/Frameworks/librococoa.dylib
--prepared:/Applications/Cyberduck.app/Contents/Frameworks/libjdns_sd.dylib
--validated:/Applications/Cyberduck.app/Contents/Frameworks/libjdns_sd.dylib
--prepared:/Applications/Cyberduck.app/Contents/Library/Spotlight/Cyberduck Spotlight Importer.mdimporter
--prepared:/Applications/Cyberduck.app/Contents/Frameworks/libjnidispatch.dylib
--validated:/Applications/Cyberduck.app/Contents/Frameworks/libjnidispatch.dylib
--prepared:/Applications/Cyberduck.app/Contents/Frameworks/libcore.dylib
--validated:/Applications/Cyberduck.app/Contents/Frameworks/libcore.dylib
--prepared:/Applications/Cyberduck.app/Contents/Frameworks/JavaNativeFoundation.framework/Versions/Current/.
--validated:/Applications/Cyberduck.app/Contents/Library/Spotlight/Cyberduck Spotlight Importer.mdimporter
--validated:/Applications/Cyberduck.app/Contents/Frameworks/JavaNativeFoundation.framework/Versions/Current/.
./Cyberduck: valid on disk
./Cyberduck: satisfies its Designated Requirement
matthew@MacBookPro /A/C/C/MacOS [1]> spctl -a -vv ./Cyberduck
./Cyberduck: accepted
source=Mac App Store
origin=Apple Mac OS Application Signing
matthewberryman commented 2 months ago

So starting it from the CLI, I get an exit status of 173, https://github.com/iterate-ch/cyberduck/blob/b31156b6af5579c1274355afe5f58d03e796e7fc/core/src/main/java/ch/cyberduck/core/aquaticprime/ReceiptFactory.java#L40 which at the GUI level obviously flags it as "corrupt"

The error message I was seeing in the logs is coming from here: https://github.com/iterate-ch/cyberduck/blob/b31156b6af5579c1274355afe5f58d03e796e7fc/core/src/main/java/ch/cyberduck/core/aquaticprime/ReceiptVerifier.java#L194

Here you are using the MAC address: https://github.com/iterate-ch/cyberduck/blob/b31156b6af5579c1274355afe5f58d03e796e7fc/core/src/main/java/ch/cyberduck/core/aquaticprime/ReceiptVerifier.java#L162

But that won't work for macOS 15, since it randomises the MAC address: https://www.macrumors.com/2024/06/10/ios-18-rotate-wifi-address/

(Note: this feature is turned off for my workplace wifi network, but maybe macOS 15 uses a different MAC address per wifi network, or has changed it on upgrade, I will explore when I get home)

Maybe the hardware UUID can be used instead, but that introduces a couple of issues:

  1. Migration of users from MAC address to hardware UUID.
  2. Logic board replacements / change of computer (although that would have been an issue with the MAC based way, I'm guessing).
dkocher commented 2 months ago

@matthewberryman Thanks for looking into this in detail. I assume this will break many applications as the sample code ^1 by Apple uses en0 interface as well for validation. I have filed FB13979956 as I don't know how the implementation would need to be changed.

There is some documentation in ^2 not yet updated for macOS 15.

matthewberryman commented 2 months ago

@dkocher Ah, the <sarcasm>joy</sarcasm> of being an Apple developer. I'm really curious as to what Apple come back to on FB13979956, as you point out the docs themselves use the MAC address, which is now no longer fixed... Further up the on-device validation page there's a note suggesting the use of AppTransaction, but:

  1. Only for macOS 13+
  2. Only in swift as far as I can see, so one would have to write a wrapper class in swift, then wrap that up in objective-c, then use JNI to get from there to Java (!). There must be an easier path...
matthewberryman commented 2 months ago

So, although I don't have the MAC address randomisation turned on for my home wifi, I do get a different MAC address.

Forcing this line to if (true) { when building from source (to force the behaviour we want to see), when the app is opened, gives a window with the following message

exit(173) Not Available

The exit(173) API is no longer available. 
You can use Transaction.all or AppTransaction.shared to verify in‑app purchases instead.

before a subsequent window with the message:

“/Users/matthew/code/cyberduck/osx/target/Cyberduck.app” is damaged and can’t be opened. 
Delete “/Users/matthew/code/cyberduck/osx/target/Cyberduck.app” and download it again from the App Store.

There's a swift example here although then you'd need to introduce swift code and expose it by @objc public class... and @objc public func... and then drop in a .h file to expose the interface to obj-c and then do JNI and then Java. Introducing swift would bump up the minimum macOS required from 10.7 to 10.9.

zeemyself commented 2 months ago
Screenshot 2024-06-27 at 9 42 06 PM

Also affect Mountain Duck downloaded from Mac App Store

devinbaeten commented 1 month ago

+1 for seeing this issue with mountain duck when downloaded from App Store:

SCR-20240813-oqxf
xhruso00 commented 3 weeks ago

I have just tested Apple sample code and it gives correct "legacy" mac address (works out of the box). The issue is the implementation of en0.getHardwareAddress(); which gives the new randomized mac address.

ioreg on Sequoia gives "IOMACAddress" = <9e36f934XXXX> for AppleBCMWLANSkywalkInterface and "IOMACAddress" = <bcd0744fXXXX> for IOSkywalkLegacyEthernet. This used to be the same value on Sonoma.

Here is the output on macOS Sequoia

MacBook-Pro ~ % ioreg -rl -c AppleBCMWLANSkywalkInterface
+-o AppleBCMWLANSkywalkInterface  <class IOUserNetworkWLAN, id 0x100000e93, registered, matched, active, busy 0 (71 ms), retain 25>
  | {
  |   "kOSBundleDextUniqueIdentifier" = <bf4014f126adca6158d1b91a459f6ba06bff27b96c102b1c09a372baad64d15e>
  |   "IO80211BSSID" = <c0f6ec1a4ad4>
  |   "IO80211RSNDone" = Yes
  |   "IOInterfaceType" = 6
  |   "IO80211ChannelBandwidth" = 80
  |   "IOUserServerPreserveUserspaceReboot" = Yes
  |   "IOUserServerOneProcess" = Yes
  |   "IO80211CountryCode" = "MY"
  |   "mDNS_Keepalive" = Yes
  |   "IO80211SSID" = "TIME_MH"
  |   "IO80211InterfaceRole" = "Infrastructure"
  |   "IOPropertyMatch" = {"IOUserClass"="AppleBCMWLANCore"}
  |   "IOServiceDEXTEntitlements" = "com.apple.developer.driverkit.family.networking"
  |   "IOUserClasses" = ("AppleBCMWLANSkywalkInterface","AppleBCMWLANInfraProtocol","IO80211InfraProtocol","IO80211InfraInterface","IO80211SkywalkInterface","IOUserNetworkWLAN","IOUserNetworkEthernet","IOService","OSObject")
  |   "IOInterfaceNamePrefix" = "en"
  |   "IOUserServerCDHash" = "8f350ddbe6b9c5eb2b9b998fb2dc6452673c1207"
  |   "built-in" = <00>
  |   "IOPersonalityPublisher" = "com.apple.DriverKit-AppleBCMWLAN"
  |   "IOPowerManagement" = {"DevicePowerState"=2,"CurrentPowerState"=2,"CapabilityFlags"=2,"MaxPowerState"=2}
  |   "mDNS_KEY" = "2009-07-30"
  |   "IO80211ChannelFrequency" = 5180
  |   "IOUserServerName" = "com.apple.bcmwlan"
  |   "IOMACAddress" = <9e36f934XXXX>
  |   "IO80211HardwareVersion" = "vendorid: 0x14e4 deviceid: 0x4433 radiorev: 0xc50dd chipnum: 0x4387 chiprev: 0x7 corerev: 0x55 boardid: 0x93c boardvendor: 0x14e4 boardrev: 0x1200 driverrev: 0x0 ucoderev: 0x5c60064 bus: 0x0 "
  |   "IOMatchedPersonality" = {"IOClass"="IOUserNetworkWLAN","CFBundleIdentifier"="com.apple.DriverKit-AppleBCMWLAN","IOProviderClass"="IOUserService","IOPropertyMatch"={"IOUserClass"="AppleBCMWLANCore"},"IO80211InterfaceRole"="Infrastru$
  |   "IO80211Channel" = 36
  |   "CFBundleIdentifier" = "com.apple.DriverKit-AppleBCMWLAN"
  |   "IOInterfaceName" = "en0"
  |   "IO80211APIUserClientProperties" = {"IOClass"="IOUserUserClient","IOUserClass"="IO80211APIUserClient"}
  |   "IOProviderClass" = "IOUserService"
  |   "IOUserClass" = "AppleBCMWLANSkywalkInterface"
  |   "IOPMResetPowerStateOnWake" = Yes
  |   "IO80211DriverVersion" = "wl0: Jul 26 2024 20:03:40 version 20.10.1123.2.8.7.186 FWID 01-94146b8e"
  |   "IOInterfaceUnit" = 0
  |   "CFBundleIdentifierKernel" = "com.apple.iokit.IOSkywalkFamily"
  |   "IOClass" = "IOUserNetworkWLAN"
  |   "IO80211Band" = "5 GHz"
  |   "IO80211Locale" = "Unknown"
  |   "IOMatchCategory" = "IODefaultMatchCategory"
  |   "IOProbeScore" = 0
  | }
  | 
  +-o IOSkywalkLegacyEthernet  <class IOSkywalkLegacyEthernet, id 0x100000f10, !registered, !matched, active, busy 0 (1 ms), retain 8>
  | | {
  | |   "IOClass" = "IOSkywalkLegacyEthernet"
  | |   "CFBundleIdentifier" = "com.apple.iokit.IOSkywalkFamily"
  | |   "IOProviderClass" = "IOSkywalkEthernetInterface"
  | |   "IOPacketFilters" = {"IONetworkFilterGroup"=275,"IOEthernetWakeOnLANFilterGroup"=0}
  | |   "IOMACAddress" = <bcd0744fXXXX>
  | |   "IOLinkSpeed" = 0
  | |   "IOProbeScore" = 0
  | |   "IOMatchedAtBoot" = Yes
  | |   "AVBControllerState" = 1
  | |   "IOMatchCategory" = "IOSkywalkLegacyEthernet"
  | |   "IOSelectedMedium" = ""
  | |   "IOClassNameOverride" = "IO80211Controller"
  | |   "IOPersonalityPublisher" = "com.apple.iokit.IOSkywalkFamily"
  | |   "IOFeatures" = 0
  | |   "CFBundleIdentifierKernel" = "com.apple.iokit.IOSkywalkFamily"
  | |   "IOLinkStatus" = 3
  | |   "IOMaxPacketSize" = 1518
  | |   "IOActiveMedium" = ""
  | |   "IOMinPacketSize" = 64
  | | }
  | | 
  | +-o en0  <class IOSkywalkLegacyEthernetInterface, id 0x100000f12, registered, matched, active, busy 0 (1 ms), retain 13>
  |   | {
  |   |   "IOLocation" = ""
  |   |   "IORequiredPacketFilters" = {"IONetworkFilterGroup"=3,"IOEthernetWakeOnLANFilterGroup"=0}
  |   |   "IOLinkActiveCount" = 0
  |   |   "BSD Name" = "en0"
  |   |   "IOMaxTransferUnit" = 1500
  |   |   "IOInterfaceType" = 6
  |   |   "IOInterfaceFlags" = 2082
  |   |   "IOMediaAddressLength" = 6
  |   |   "IOInterfaceState" = 3
  |   |   "IOMediaHeaderLength" = 14
  |   |   "IOActivePacketFilters" = {"IONetworkFilterGroup"=0,"IOEthernetWakeOnLANFilterGroup"=0}
  |   |   "IOInterfaceExtraFlags" = 0
  |   |   "IOPrimaryInterface" = Yes
  |   |   "IOInterfaceUnit" = 0
  |   |   "IOInterfaceNamePrefix" = "en"
  |   |   "IOBuiltin" = Yes
  |   |   "IONetworkData" = {"IONetworkStatsKey"={"Size"=20,"Data"=<0000000000000000000000000000000000000000>,"Access Types"=9},"IOEthernetStatsKey"={"Size"=216,"Data"=<000000000000000000000000000000000000000000000000000000000000000000000$
  |   | }
  |   |

Here is the output of ioreg on Sonoma

MacBook-Pro-M1 ~ % ioreg -rl -c AppleBCMWLANSkywalkInterface
+-o AppleBCMWLANSkywalkInterface  <class AppleBCMWLANSkywalkInterface, id 0x100000a7f, registered, matched, active, busy 0 (1115 $
  | {
  |   "mDNS_KEY" = "2009-07-30"
  |   "mDNS_Keepalive" = Yes
  |   "IO80211Channel" = 36
  |   "IO80211Band" = "5 GHz"
  |   "IO80211BSSID" = <000000000000>
  |   "IO80211InterfaceRole" = "Infrastructure"
  |   "IO80211ChannelBandwidth" = 80
  |   "IO80211SSID" = ""
  |   "IOPowerManagement" = {"ChildrenPowerState"=1,"DevicePowerState"=1,"CurrentPowerState"=1,"CapabilityFlags"=32770,"MaxPowerS$
  |   "built-in" = <00>
  |   "IO80211DriverVersion" = "wl0: May 13 2024 23:59:32 version 20.103.15.0.8.7.175 FWID 01-fd39ee3a"
  |   "IOInterfaceName" = "en0"
  |   "IOInterfaceUnit" = 0
  |   "IO80211Locale" = "ETSI"
  |   "IOMACAddress" = <bcd0744fXXXX>
  |   "IOInterfaceType" = 6
  |   "IOPMResetPowerStateOnWake" = Yes
  |   "IO80211RSNDone" = Yes
  |   "IO80211CountryCode" = "MY"
  |   "IO80211ChannelFrequency" = 5180
  |   "IO80211HardwareVersion" = "vendorid: 0x14e4 deviceid: 0x4433 radiorev: 0xc50dd chipnum: 0x4387 chiprev: 0x7 corerev: 0x55 $
  |   "IOInterfaceNamePrefix" = "en"
  | }
  | 
  +-o IOSkywalkLegacyEthernet  <class IOSkywalkLegacyEthernet, id 0x100000a8c, !registered, !matched, active, busy 0 (0 ms), reta$
  | | {
  | |   "IOClass" = "IOSkywalkLegacyEthernet"
  | |   "CFBundleIdentifier" = "com.apple.iokit.IOSkywalkFamily"
  | |   "IOProviderClass" = "IOSkywalkEthernetInterface"
  | |   "IOPacketFilters" = {"IONetworkFilterGroup"=275,"IOEthernetWakeOnLANFilterGroup"=0}
  | |   "IOMACAddress" = <bcd0744fXXXX>
  | |   "IOLinkSpeed" = 0
  | |   "IOProbeScore" = 0
  | |   "IOMatchedAtBoot" = Yes
  | |   "AVBControllerState" = 1
  | |   "IOMatchCategory" = "IOSkywalkLegacyEthernet"
  | |   "IOSelectedMedium" = ""
  | |   "IOClassNameOverride" = "IO80211Controller"
  | |   "IOPersonalityPublisher" = "com.apple.iokit.IOSkywalkFamily"
  | |   "IOFeatures" = 0
  | |   "CFBundleIdentifierKernel" = "com.apple.iokit.IOSkywalkFamily"
  | |   "IOLinkStatus" = 3
  | |   "IOMaxPacketSize" = 1518
  | |   "IOActiveMedium" = ""
  | |   "IOMinPacketSize" = 64
  | | }
  | | 
  | +-o en0  <class IOSkywalkLegacyEthernetInterface, id 0x100000a8e, registered, matched, active, busy 0 (0 ms), retain 12>
  |   | {
  |   |   "IOLocation" = ""
  |   |   "IORequiredPacketFilters" = {"IONetworkFilterGroup"=3,"IOEthernetWakeOnLANFilterGroup"=0}
  |   |   "BSD Name" = "en0"
  |   |   "IOMaxTransferUnit" = 1500
  |   |   "IOInterfaceFlags" = 2082
  |   |   "IOInterfaceType" = 6
  |   |   "IOMediaAddressLength" = 6
  |   |   "IOInterfaceState" = 3
  |   |   "IOMediaHeaderLength" = 14
  |   |   "IOActivePacketFilters" = {"IONetworkFilterGroup"=0,"IOEthernetWakeOnLANFilterGroup"=0}
  |   |   "IOInterfaceExtraFlags" = 0
  |   |   "IOPrimaryInterface" = Yes
  |   |   "IOInterfaceUnit" = 0
  |   |   "IOInterfaceNamePrefix" = "en"
  |   |   "IOBuiltin" = Yes
  |   |   "IONetworkData" = {"IONetworkStatsKey"={"Size"=20,"Data"=<0000000000000000000000000000000000000000>,"Access Types"=9},"$
  |   | }
  |   |