Support package 3.9-b2 is unstable

[freakboy3742]

Describe the bug

Support package 3.9-b2 segfaults on iOS.

To Reproduce

Steps to reproduce the behavior:

1. Build and Run a helloworld Toga app on iOS

Expected behavior

App starts.

Actual behavior is a segfault when the Toga app is instantiated - EXC_BAD_ACCESS (code=2, address=0x107004d30); top stack frame is:

Hello World`PyObject_IS_GC:
    0x103dd79f0 <+0>:  pushq  %rbp
    0x103dd79f1 <+1>:  movq   %rsp, %rbp
    0x103dd79f4 <+4>:  movq   0x8(%rdi), %rcx
    0x103dd79f8 <+8>:  xorl   %eax, %eax
    0x103dd79fa <+10>: testb  $0x40, 0xa9(%rcx)
    0x103dd7a01 <+17>: je     0x103dd7a1a               ; <+42> at gcmodule.c:2207:5
    0x103dd7a03 <+19>: movq   0x148(%rcx), %rax
    0x103dd7a0a <+26>: testq  %rax, %rax
    0x103dd7a0d <+29>: je     0x103dd7a1c               ; <+44> at gcmodule.c
    0x103dd7a0f <+31>: callq  *%rax
->  0x103dd7a11 <+33>: movl   %eax, %ecx
    0x103dd7a13 <+35>: xorl   %eax, %eax
    0x103dd7a15 <+37>: testl  %ecx, %ecx
    0x103dd7a17 <+39>: setne  %al
    0x103dd7a1a <+42>: popq   %rbp
    0x103dd7a1b <+43>: retq   
    0x103dd7a1c <+44>: movl   $0x1, %eax
    0x103dd7a21 <+49>: popq   %rbp
    0x103dd7a22 <+50>: retq   

Environment:

• Operating System: iOS 14.4 iPhone SE 2nd Gen simulator
• Python version: 3.9.2 (3.9-b2 support package)
• Software versions:
  • Briefcase: 0.3.5
  • Toga: 0.3.0dev25
  • Rubicon: 0.4.0 and dev 7b1e109473

Additional context

I've tracked the problem as far as the definition of Rubicon's DeallocationObserver. Basic registration of NSObject, NSNumber etc classes appears to work; however, defining the DeallocationObserver fails. The problem occurs on both Rubicon 0.4.0 and HEAD, so it doesn't appear to be related to the recent changes around memory management.

Support package 3.9-b1, as well as a custom build of the 3.9 branch as of b0d860cc, both work without error. The problem appears to have been introduced when the support package was updated to Python 3.9.2.

This was back ported to 3.8.8; however, 3.8-b5 appears to work.

[freakboy3742]

@dgelessus Don't know if you have any spare time at the moment, but if you do - any help you can provide here would be most appreciated.

[dgelessus]

So I haven't actually built and tested it myself yet, because I don't have a simulator or device set up for development right now.

I had a look over the changes between Python 3.9.1 and 3.9.2 - the only commits that stood out to me were python/cpython@60463e8e4f79e5b5e96dc43fb83ded373b489e33 (because it changes something related to garbage collection) and python/cpython@7e729978fa08a360cbf936dc215ba7dd25a06a08 (because it touches memory management code related to libffi). It might be worth a try to revert either of those two commits and see if that prevents the crash. If not, then I don't really have any ideas other than git bisecting all changes between 3.9.1 and 3.9.2. (I'm assuming that this is in fact a CPython issue, because it happens only exactly after the update to 3.9.2 and isn't affected by the exact version of rubicon-objc.)

[freakboy3742]

@dgelessus Thanks for that investigation - I guess it's time for me to go spelunking :-)

[freakboy3742]

The problem was related to https://github.com/python/cpython/pull/23868 - but the issue was on our end. We were enabling some preprocessor flags (HAVE_FFI_CLOSURE_ALLOC and HAVE_FFI_PREP_CIF_VAR) that weren't required, but with the code change in CPython 3.9.2, enabled some code that caused the crash we were seeing.

[freakboy3742]

Resolved with the publication of 3.9-b3.