Crash on macOS Big Sur 11.4

[cheerlessDreamer]

I get the following error when using darkdetect from the terminal:

>>> darkdetect.isDark()
2021-06-08 19:36:55.831 Python[4299:381204] +[NSUserDefaults standardUserDefaults]: unrecognized selector sent to class 0x7fff8011ea10
2021-06-08 19:36:55.835 Python[4299:381204] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '+[NSUserDefaults standardUserDefaults]: unrecognized selector sent to class 0x7fff8011ea10'
*** First throw call stack:
(
    0   CoreFoundation                      0x00007fff2055287b __exceptionPreprocess + 242
    1   libobjc.A.dylib                     0x00007fff2028ad92 objc_exception_throw + 48
    2   CoreFoundation                      0x00007fff205d52e5 __CFExceptionProem + 0
    3   CoreFoundation                      0x00007fff204ba90b ___forwarding___ + 1448
    4   CoreFoundation                      0x00007fff204ba2d8 _CF_forwarding_prep_0 + 120
    5   libffi.dylib                        0x00007fff2d9ba8f5 ffi_call_unix64 + 85
    6   ???                                 0x00007ffeee3d67e0 0x0 + 140732895422432
)
libc++abi: terminating with uncaught exception of type NSException
zsh: abort      python3

This happens when using any method, i.e., isDark, isLight, or theme. For information, I'm running macOS Big Sur 11.4 (20F71).

[Ahsoka]

Hmmmm... interesting, I also have macOS Big Sur 11.4, however, no crash when I run this. However, despite being in dark mode, it still states that I am in light mode.

(Using Intel Mac)

See this photo for reference:

[MTaylor-tech]

Also on Big Sur:

Python 3.9.5 (default, Aug 20 2021, 17:29:18) [Clang 12.0.5 (clang-1205.0.22.11)] on darwin Type "help", "copyright", "credits" or "license" for more information.

import darkdetect print(darkdetect.theme()) [1] 87474 segmentation fault python

[Ahsoka]

Also on Big Sur:

Python 3.9.5 (default, Aug 20 2021, 17:29:18) [Clang 12.0.5 (clang-1205.0.22.11)] on darwin Type "help", "copyright", "credits" or "license" for more information.

import darkdetect print(darkdetect.theme()) [1] 87474 segmentation fault python

Are you using an M1 Mac or Intel Mac?

[Ahsoka]

Also could you trying running this code, for a bit more debugging info?

import faulthandler
faulthandler.enable()

import darkdetect
print(darkdetect.theme())

[psobolewskiPhD]

I'm on Big Sur 11.5.2, but M1 MBP and having this same issue. I used the faulthandler as instructed above in ipython:

Python 3.9.6 | packaged by conda-forge | (default, Jul  6 2021, 08:51:19) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.25.0 -- An enhanced Interactive Python. Type '?' for help.

Fatal Python error: Segmentation fault

Thread 0x00000001705db000 (most recent call first):
  File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/threading.py", line 312 in wait
  File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/threading.py", line 574 in wait
  File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/core/history.py", line 829 in run
  File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/core/history.py", line 58 in needs_sqlite
  File "<decorator-gen-17>", line 2 in run
  File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/threading.py", line 973 in _bootstrap_inner
  File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/threading.py", line 930 in _bootstrap

Current thread 0x0000000100f43d40 (most recent call first):
  File "/Users/piotrsobolewski/Dev/napari/napari/_vendor/darkdetect/_mac_detect.py", line 44 in theme
  File "<ipython-input-3-30534ba1feb8>", line 1 in <module>
  File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3441 in run_code
  File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3361 in run_ast_nodes
  File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 3169 in run_cell_async
  File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/core/async_helpers.py", line 68 in _pseudo_sync_runner
  File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 2944 in _run_cell
  File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/core/interactiveshell.py", line 2898 in run_cell
  File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/terminal/interactiveshell.py", line 555 in interact
  File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/terminal/interactiveshell.py", line 564 in mainloop
  File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/terminal/ipapp.py", line 356 in start
  File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/traitlets/config/application.py", line 845 in launch_instance
  File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/lib/python3.9/site-packages/IPython/__init__.py", line 126 in start_ipython
  File "/Users/piotrsobolewski/Dev/miniforge3/envs/napari-test/bin/ipython", line 8 in <module>
fish: Job 1, 'ipython' terminated by signal SIGSEGV (Address boundary error)

[psobolewskiPhD]

Wonder if this is related to the fact that running: defaults read -globalDomain AppleInterfaceStyle from the Terminal on Big Sur returns an error while in Light mode:

2021-09-11 19:39:43.262 defaults[63934:11811421] 
The domain/default pair of (kCFPreferencesAnyApplication, AppleInterfaceStyle) does not exist

The same command, when run in Dark mode returns: Dark

[Ahsoka]

Hmmm... interesting, perhaps using that command would be more reliable than using ctypes. Perhaps something like:

import subprocess

def theme():
    sub = subprocess.Popen("defaults read -globalDomain AppleInterfaceStyle".split(), stdout=subprocess.PIPE)
    result = sub.stdout.read()
    if isinstance(result, bytes):
        try:
            result = result.decode()
        except UnicodeDecodeError:
            return 'Light'
        if result[-1] == '\n':
            result = result[:-1]
        return 'Dark' if result == 'Dark' else 'Light'

Not sure how this works with other versions of macOS though. Works well on my Intel Mac running Big Sur 11.4.

[psobolewskiPhD]

I've used this python hack of a terminal command:

import os
test = os.popen('defaults read -globalDomain AppleInterfaceStyle &> /dev/null && echo dark || echo light')
theme = test.read().strip()

Works on Big Sur M1.

Edit: Can report your code above using subprocess works in Dark mode, but not light:

In [3]: theme()
Out[3]: 'Dark'
<<<switch to light>>>
In [4]: theme()
2021-09-11 22:52:05.102 defaults[65849:11909460] 
The domain/default pair of (kCFPreferencesAnyApplication, AppleInterfaceStyle) does not exist
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-5-8bbb2e4d564a> in <module>
----> 1 theme()

<ipython-input-2-b9ad10b0edfb> in theme()
      7         except UnicodeDecodeError:
      8             return 'Light'
----> 9         if result[-1] == '\n':
     10             result = result[:-1]
     11         return 'Dark' if result == 'Dark' else 'Light'

IndexError: string index out of range

[albertosottile]

Thanks to all for reporting and investigating this. I have also experienced this issue on Big Sur 11.3+ and an Intel Mac.

What I find intriguing is that the issue occurs only when running the code from Terminal. If the code is called by a proper .app bundle (e.g. one frozen with py2app), then the detection always works as expected.

To me, this and the error message we see ([NSUserDefaults standardUserDefaults]: unrecognized selector sent to class) point to some context missing in the Terminal app. Perhaps we need to create this context manually In darkdetect to make sure the detection works also from Terminal. Alas, I have no idea how. Perhaps some Cocoa developer can jump in.

@psobolewskiPhD @Ahsoka Detection methods based on command invocations + subprocess usually fails when executed from a frozen application (e.g. a py2app bundle). I coded darkdetect with ctypes precisely to be able to detect the Dark mode from a frozen Python application.

[Ahsoka]

Maybe when not frozen you can use a command invocation for detection and when frozen use the ctypes approach.

Psuedo code:

import sys

def theme():
    if getattr(sys, 'frozen', False):
        ctypes_method()
    else:
        command_invocation_method()

[albertosottile]

I found the issue: libobjc.dylib was not correctly loaded by ctypes due to a change in Big Sur:

New in macOS Big Sur 11 beta, the system ships with a built-in dynamic linker cache of all system-provided libraries. As part of this change, copies of dynamic libraries are no longer present on the filesystem. Code that attempts to check for dynamic library presence by looking for a file at a path or enumerating a directory will fail. Instead, check for library presence by attempting to dlopen () the path, which will correctly check for the library in the cache. (62986286)

I now wonder how this worked from .app bundles, perhaps ctypes.util.find_library is able to locate library files when called from there.

Patching https://github.com/albertosottile/darkdetect/blob/aa8a78ecba455e7532d962d8f7eb186b88217149/darkdetect/_mac_detect.py#L10-L11 to

appkit = ctypes.cdll.LoadLibrary('AppKit.framework/AppKit')
objc = ctypes.cdll.LoadLibrary('libobjc.dylib')

allows me to correctly run the package from Terminal on Big Sur 11.3.1.

I assume the change is not compatible with older macOS versions so, a proper fix will require to retain the old code for backward compatibility. Also, the effects of this change to .app bundles needs to be investigated. I intend to do this as soon as I have some spare time, then release a new version of darkdetect containing this patch.

Thanks again to all the people that contributed to this discussion.

(Credits for the help I found in https://github.com/vispy/vispy/issues/1885 and https://github.com/vispy/vispy/pull/1975).

[albertosottile]

Fixed in 0.4.0.

[psobolewskiPhD]

Alas, the issue is not resolved for me Big Sur 11.5.2 but on M1 mac

Collecting darkdetect
  Downloading darkdetect-0.4.0-py3-none-any.whl (6.3 kB)
Installing collected packages: darkdetect
Successfully installed darkdetect-0.4.0
╭─ ~ ···································································· ✔ ─╮
╰─ ipython                                                      (napari-test) ─╯
Python 3.9.6 | packaged by conda-forge | (default, Jul  6 2021, 08:51:19) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.25.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import darkdetect

In [2]: darkdetect.theme()
fish: Job 1, 'ipython' terminated by signal SIGSEGV (Address boundary error)

Edit: I tested in both light and dark mode.

[Ahsoka]

Hmmm... very interesting, the issue is resolved for me on my Intel Mac and it appears that @albertosottile also has an Intel Mac. Seems like a M1 Mac specific issue.

[psobolewskiPhD]

That's what I feared 🦑 I thought it was related to 3.9.6 vs 3.9.7, so I updated python, but same:

╰─ ipython                                                      (napari-test) ─╯
Python 3.9.7 | packaged by conda-forge | (default, Sep  2 2021, 17:55:16) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.25.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import darkdetect

In [2]: darkdetect.theme()
fish: Job 1, 'ipython' terminated by signal SIGSEGV (Address boundary error)

[albertosottile]

@psobolewskiPhD It appears we have two different issues at hand. Unfortunately, I do not have a M1 Mac to reproduce/debug this one. However, in this case I would ask you to open another issue on GitHub to better distinguish the latter bug from the one that I just patched.

In case you want to do that, it might be a good idea to remove conda and ipython from the equation, and run directly darkdetect from the system Python interpreter (perhaps after importing faulthandler as suggested by @Ahsoka ). If you do not want to install the package, you can run the test script by just checking out the git repository, as this package has no requirements.

[psobolewskiPhD]

I think I've narrowed it down. I think it's related to the objc_msgSend, based on this: https://developer.apple.com/documentation/apple-silicon/addressing-architectural-differences-in-your-macos-code At the top we have: objc.objc_msgSend.argtypes = [void_p, void_p]

But in def theme(): appearanceNS = msg(stdUserDef, n('stringForKey:'), void_p(key))

This triggers segfault.

You can do a simple test by using:

NSString = C('NSString')

objc.objc_msgSend(NSString, n("stringWithUTF8String:"), _utf8("reason"))

If the .argtypes is just [void_p, void_p] it segfaults, if it's [void_p, void_p, void_p] it runs.

I've fixed it locally, by making the .argtypes have 3 arguments and then passing None as the third everywhere. Not very elegant, but it works. I can open issue/PR if you think that's a way forward.

BTW, not sure why but at least on my system (now 3.9.7) the new V check code isn't needed. This works just fine:

appkit = ctypes.cdll.LoadLibrary(ctypes.util.find_library('AppKit'))
objc = ctypes.cdll.LoadLibrary(ctypes.util.find_library('objc'))

Edit: just to show:

appkit = ctypes.cdll.LoadLibrary(ctypes.util.find_library('AppKit'))

output:

<CDLL '/System/Library/Frameworks/AppKit.framework/AppKit', handle 115edc6f0 at 0x1075a6850>

[albertosottile]

@psobolewskiPhD I'd definitely open a new issue and report/resume this discussion there.

[albertosottile]

@psobolewskiPhD Re: the need of the "V" patch. I suspect eventually the Python teams would integrate/had integrated the patch within ctypes itself, which is probably what happened in 3.9.7. Nevertheless, I would like to retain compatibility with older minor versions, hence the need to add this patch.