macOS Tk failing from release 20220222 onwards.

tmontes commented 2 years ago

Hey @indygreg,

It's me again, exploring your wonderful project, and experiencing a tkinter failure on macOS.

My minimal test:

import tkinter
root = tkinter.Tk()

Works up to 20220222. On that release, with 3.10.2, fails with:

Python 3.10.2 (main, Feb 20 2022, 20:00:41) [Clang 13.0.1 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tkinter
>>> root = tkinter.Tk()
2022-06-21 21:28:57.868 PuPPY[81171:2337557] -[TKApplication tkProcessEvent:]: unrecognized selector sent to instance 0x7f958b774200
2022-06-21 21:28:57.871 PuPPY[81171:2337557] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[TKApplication tkProcessEvent:]: unrecognized selector sent to instance 0x7f958b774200'
*** First throw call stack:
(
    0   CoreFoundation                      0x00007fff2d05e035 __exceptionPreprocess + 256
    1   libobjc.A.dylib                     0x00007fff578c6a17 objc_exception_throw + 48
    2   CoreFoundation                      0x00007fff2d0d7fce -[NSObject(NSObject) __retain_OA] + 0
    3   CoreFoundation                      0x00007fff2cfffe9f ___forwarding___ + 1485
    4   CoreFoundation                      0x00007fff2cfff848 _CF_forwarding_prep_0 + 120
    5   libpython3.10.dylib                 0x000000010f6a9ca7 TkMacOSXEventsCheckProc + 351
    6   libpython3.10.dylib                 0x000000010f5ab4cf Tcl_DoOneEvent + 306
    7   libpython3.10.dylib                 0x000000010f6a1ede TkpInit + 389
    8   libpython3.10.dylib                 0x000000010f62130c Initialize + 2046
    9   libpython3.10.dylib                 0x000000010f97471c Tkapp_New + 924
    10  libpython3.10.dylib                 0x000000010f9740a0 _tkinter_create + 640
    11  libpython3.10.dylib                 0x000000010f76ce3b cfunction_vectorcall_FASTCALL + 171
    12  libpython3.10.dylib                 0x000000010f81e123 call_function + 419
    13  libpython3.10.dylib                 0x000000010f819f7f _PyEval_EvalFrameDefault + 28159
    14  libpython3.10.dylib                 0x000000010f8129fa _PyEval_Vector + 2362
    15  libpython3.10.dylib                 0x000000010f71ac06 _PyObject_FastCallDictTstate + 262
    16  libpython3.10.dylib                 0x000000010f71b6c8 _PyObject_Call_Prepend + 152
    17  libpython3.10.dylib                 0x000000010f79084f slot_tp_init + 207
    18  libpython3.10.dylib                 0x000000010f788ed4 type_call + 324
    19  libpython3.10.dylib                 0x000000010f71a86d _PyObject_MakeTpCall + 253
    20  libpython3.10.dylib                 0x000000010f81e241 call_function + 705
    21  libpython3.10.dylib                 0x000000010f819f7f _PyEval_EvalFrameDefault + 28159
    22  libpython3.10.dylib                 0x000000010f8129fa _PyEval_Vector + 2362
    23  libpython3.10.dylib                 0x000000010f87267b PyRun_InteractiveOneObjectEx + 1227
    24  libpython3.10.dylib                 0x000000010f8714eb _PyRun_InteractiveLoopObject + 427
    25  libpython3.10.dylib                 0x000000010f870ae7 _PyRun_AnyFileObject + 103
    26  libpython3.10.dylib                 0x000000010f8743c8 PyRun_AnyFileExFlags + 168
    27  libpython3.10.dylib                 0x000000010f896114 Py_RunMain + 2788
    28  libpython3.10.dylib                 0x000000010f8973a1 pymain_main + 1681
    29  libpython3.10.dylib                 0x000000010f8977eb Py_BytesMain + 43
    30  libdyld.dylib                       0x00007fff590963d5 start + 1
    31  ???                                 0x0000000000000001 0x0 + 1
)
libc++abi.dylib: terminating with uncaught exception of type NSException
Abort trap: 6

What else I tested:

Failing -- macOS 10.14.6:
- 3.10 pgo, lto, pgo+lto, and debug builds.
- 3.9 pgo and lto builds.
- 3.10 pgo build of the most recent release, 20220528.
Working -- CentOS 7 Linux and Windows, both x86-64:
- 3.10 build of most recent release.

Release notes for 20220222 indicate that Tcl/Tk was upgraded from 8.6.10 to 8.6.12, so this provides at least some explanation to my observations.

Can you please help in getting Tk back in shape on macOS? How can I assist in the process?

Again, thanks a lot for sharing the project and, in advance, for helping address this.

indygreg commented 2 years ago

Thanks for the report!

I'm able to reproduce this on an M1 using macOS 12. That seemingly rules out a macOS targeting bug. My stack from a REPL is slightly different:

 python/bin/python3
Python 3.10.4 (main, May 28 2022, 14:22:07) [Clang 14.0.3 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tkinter
>>> root = tkinter.Tk()
>>> 2022-06-26 10:26:27.805 python3[30525:3649599] -[TKApplication tkProcessEvent:]: unrecognized selector sent to instance 0x12da807f0
2022-06-26 10:26:27.818 python3[30525:3649599] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[TKApplication tkProcessEvent:]: unrecognized selector sent to instance 0x12da807f0'
*** First throw call stack:
(
        0   CoreFoundation                      0x00000001c2ec9198 __exceptionPreprocess + 240
        1   libobjc.A.dylib                     0x00000001c2c13e04 objc_exception_throw + 60
        2   CoreFoundation                      0x00000001c2f5cf40 -[NSObject(NSObject) __retain_OA] + 0
        3   CoreFoundation                      0x00000001c2e28544 ___forwarding___ + 1764
        4   CoreFoundation                      0x00000001c2e27da0 _CF_forwarding_prep_0 + 96
        5   libpython3.10.dylib                 0x0000000101741468 TkMacOSXEventsCheckProc + 332
        6   libpython3.10.dylib                 0x0000000101647030 Tcl_DoOneEvent + 268
        7   libpython3.10.dylib                 0x0000000101a28914 EventHook + 180
        8   libpython3.10.dylib                 0x0000000101abbfb8 readline_until_enter_or_signal + 264
        9   libpython3.10.dylib                 0x0000000101abb650 call_readline + 136
        10  libpython3.10.dylib                 0x000000010178aba4 PyOS_Readline + 172
        11  libpython3.10.dylib                 0x000000010178d8d0 tok_underflow_interactive + 112
        12  libpython3.10.dylib                 0x000000010178d148 tok_get + 5048
        13  libpython3.10.dylib                 0x00000001017571d0 _PyPegen_fill_token + 56
        14  libpython3.10.dylib                 0x000000010175bd4c _PyPegen_parse + 764
        15  libpython3.10.dylib                 0x000000010175992c _PyPegen_run_parser + 24
        16  libpython3.10.dylib                 0x0000000101759e24 _PyPegen_run_parser_from_file_pointer + 184
        17  libpython3.10.dylib                 0x000000010193c03c PyRun_InteractiveOneObjectEx + 392
        18  libpython3.10.dylib                 0x000000010193b340 _PyRun_InteractiveLoopObject + 116
        19  libpython3.10.dylib                 0x000000010193ae9c _PyRun_AnyFileObject + 76
        20  libpython3.10.dylib                 0x000000010193d434 PyRun_AnyFileExFlags + 68
        21  libpython3.10.dylib                 0x000000010195be28 pymain_run_stdin + 160
        22  libpython3.10.dylib                 0x000000010195b4b4 Py_RunMain + 1108
        23  libpython3.10.dylib                 0x000000010195c614 Py_BytesMain + 40
        24  dyld                                0x00000001001c508c start + 520
)
libc++abi: terminating with uncaught exception of type NSException
zsh: abort      python/bin/python3

TkMacOSXEventsCheckProc is the common function. But that function simply proxies NSEvents to X events.

The crash seemingly occurs on this line in TkMacOSXEventsCheckProx(): NSEvent *processedEvent = [NSApp tkProcessEvent:currentEvent];.

It doesn't look like CPython changed anything major with regards to tk integration between 3.10.0 and 3.10.2. So my money is on this being some kind of regression between tk 8.6.10 and 8.6.12.

I'm not great at Obj-C debugging. But given this seemingly has to do with tk's internal Obj-C message handling for TKApplication, I think appropriate next steps are to report a bug against tk and get someone more experienced with tk's internals to look into it.

It might also be worth reproducing without a statically linked tcl/tk. While static linking should be supported, it isn't common and has a history of tickling bugs due to common assumptions that libraries are shared libraries.

indygreg commented 2 years ago

CPython 3.10.1 did upgrade tcl/tk to 8.6.12. So it is good they are running the same version as us with their official macOS installers.

I looked at the CPython tcl/tk macOS build code and didn't see any glaring differences beyond the static/shared library difference.

indygreg commented 2 years ago

I compiled tk with TK_MAC_DEBUG_EVENTS defined to get some more forensics logging. Also moved TKLog(@" event: %@", currentEvent); line in TkMacOSXEventsCheckProc() to before the crashing NSEvent *processedEvent = [NSApp tkProcessEvent:currentEvent]; line. That produced the following:

% python/install/bin/python3
Python 3.10.5 (main, Jun 26 2022, 12:07:50) [Clang 14.0.3 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tkinter
>>> tkinter.Tk()
<tkinter.Tk object .>
>>> 2022-06-26 12:10:17.922 python3[20924:3832525]    event: NSEvent: type=Kitdefined loc=(0,228) time=1102562.7 flags=0 win=0x153a65dc0 winNum=364 ctxt=0x0 subtype=4 data1=1128792064 data2=1130627072
2022-06-26 12:10:17.922 python3[20924:3832525] -[TKApplication tkProcessEvent:]: unrecognized selector sent to instance 0x142e92b50
2022-06-26 12:10:17.929 python3[20924:3832525] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[TKApplication tkProcessEvent:]: unrecognized selector sent to instance 0x142e92b50'

So it crashes on a Kitdefined event. But I think the more important takeaway is it is crashing on the first event. I think the bug here is tkProcessEvent isn't being registered with NSApp / TKApplication. Why, I'm not sure. But the answer seemingly involves macosx/tkMacOSXInit.c.

indygreg commented 2 years ago

https://core.tcl-lang.org/tk/tktview/85f316beb15108ac43b03fa6c8608e31f3ae5f92 seems interesting.

indygreg commented 2 years ago

I have a release staged with this fix at https://github.com/indygreg/python-build-standalone/releases/tag/20220630. Should hopefully remove the draft label within a few hours once PyOxidizer CI comes back clean.

tmontes commented 2 years ago

@indygreg,

Wow, thank you very much for such a speedy fix. I do confirm that Tk based applications work nicely on macOS (and Windows and Linux) now!

Amazing! :-)

indygreg / python-build-standalone

macOS Tk failing from release 20220222 onwards. #132