Signing error? 1.7.0 generating lots of console errors on macOS, 55% CPU utilization

[mobilemind]

Maybe there's a code signing error??

I'm not able to do much debugging. But I'm encountering lots of issues with the latest update. Maestral is unusable I've got no access to the UI/ All I can do is us Activity Monitor to Force quit. While I leave it running my machine is getting thousands of console message like the ones below.

Library Validation failed: Rejecting '/private/var/folders/qv/95xhv4y1363dnx03h44ws2nc0000gp/T/ffivstRo9' (Team ID: none, platform: no) for process 'Maestral(2405)' (Team ID: G34LNR8C4Y, platform: no), reason: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.

Library Validation failed: Rejecting '/private/tmp/ffiVYcjOg' (Team ID: none, platform: no) for process 'Maestral(2405)' (Team ID: G34LNR8C4Y, platform: no), reason: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.

and

Library Validation failed: Rejecting '/Users/username/ffiiQfQWc' (Team ID: none, platform: no) for process 'Maestral(2405)' (Team ID: G34LNR8C4Y, platform: no), reason: mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed.

[mobilemind]

Rolled back to 1.6.4 and everything worked fine on the same machine & account.

[dylango]

Same here. macOS 12.6.1 Python 3.9.6

[samschott]

This is very strange. I've enabled library validation for macOS binary is the 1.7.0 release, meaning that loading any unsigned libraries or libraries signed by other developers will fail with the errors that you are seeing.

This should not cause any problems since Maestral is self-contained app, and I'm not seeing any issues on macOS 13. That being said, maybe there are corner cases in which Python would extract something to a temporary directory and try to read from there.

@mobilemind, could you check who owns the files that Maestral is trying to load? If you delete them, do they get regenerated the next time you try to start Maestral?

[samschott]

Whats more, nothing in those temporary folders should be a library, they should mostly contain temporary resources files which might be required at runtime but not cause any issues.

[samschott]

@mobilemind, if you could also report the macOS version, this would be very helpful!

[Stramon1um]

Rolled back to 1.6.4 and everything worked fine on the same machine & account.

same here

[samschott]

I could reproduce this now on an Intel Mac. Looks like it indeed has something to do with hardened runtime entitlements, though the "library validation" was a bit of a red herring since Maestral does indeed not load any external libraries.

Re-enabling the "allow unsigned executable memory" entitlement allows running on Intel Macs again. While I'm not entirely certain about the underlying problem, I suspect it has something to do with how Python functions are set as callbacks for ObjC code, which appears to work differently on Apple Silicon vs Intel machines.

I've released a new version which reenables this entitlement now.

[sreilly]

I know java apps need the allow unsigned executable memory entitlement due to the JIT compilation being done at run-time. I wonder if python has a similar kind of JIT that might only be used on intel?

[samschott]

This is indeed a tricky one, and it turns out the devil is in the details - in this case in how Python callbacks for C / ObjC functions are defined using trampolines in libFFI (the library that allows us to call C / ObjC code from Python).

Digging through the libFFI-27 code from Apple, I get at least a vague idea of what is going on. On ARM Apple, libFFI is compiled with FFI_EXEC_TRAMPOLINE_TABLE = 1, which provides some alternative way of defining trampolines (https://opensource.apple.com/source/libffi/libffi-27/configure.ac.auto.html). For other architectures with !FFI_MMAP_EXEC_WRIT && !FFI_EXEC_TRAMPOLINE_TABLE, you can find the following comment in closures.c (https://opensource.apple.com/source/libffi/libffi-27/src/closures.c.auto.html):

 /* This macro indicates it may be forbidden to map anonymous memory
   with both write and execute permission.  Code compiled when this
   option is defined will attempt to map such pages once, but if it
   fails, it falls back to creating a temporary file in a writable and
   executable filesystem and mapping pages from it into separate
   locations in the virtual memory space, one location writable and
   another executable.  */

This would explain the original, very strange bug report which showed attempts to load and validate executable code from temporary files. So it appears that the "allow unsigned executable memory" entitlement is indeed required on Intel but not Apple Silicon.

You live and learn :) Apologies for everyone who ended up being a guinea pig for my hardened runtime experiments!

[samschott]

Note that the above behaviour is only problematic because Maestral, in particular the GUI, defines Python callbacks for ObjC functions, for example to react to user interactions with the UI. It even creates some ObjC classes at runtime that have method definitions living in Python land. "regular" Python code embedded in a macOS binary should not run into such issues that require special entitlements.