Segmentation fault: 11

[BlackFoundry]

All seems installed and imported properly but, when I run from ttfautohint import ttfautohint ttfautohint(in_file=fontPath, out_file=f"{fontPath[:-4]}-hinted.ttf")

I get this error message: Segmentation fault: 11

[BlackFoundry]

MacOS 12.1, Apple M1 Pro

[anthrotype]

ouch, that's not good. Could you send me the font so I can try to reproduce?

[anthrotype]

can you also try enabling the faulthandler by running python3 with the -X faulthandler option, or by setting the PYTHONFAULTHANDLER=1 environment variable and see if you get more info?

[anthrotype]

the thing is, we build universal2 wheels for mac on the CI, but we don't actually run the test suite on arm64 architecture because the CI runners are only intel x86_64 (they cross-compile for arm64 but can't run the code).

Are you able to run the test suite from your M1 mac? After pip install ttfautohint-py, from the root of the git repo run pytest tests/

[BlackFoundry]

============================================================================ test session starts =============================================================================
platform darwin -- Python 3.10.2, pytest-6.2.5, py-1.11.0, pluggy-1.0.0 -- /Users/jeremiehornus/GitHub/ttfautohint-py/venv/bin/python3
cachedir: .pytest_cache
rootdir: /Users/jeremiehornus/GitHub/ttfautohint-py, configfile: setup.cfg
collected 92 items                                                                                                                                                           

tests/test_info.py::TestMutableByteString::test_tobytes[non-empty] PASSED                                                                                              [  1%]
tests/test_info.py::TestMutableByteString::test_tobytes[empty] PASSED                                                                                                  [  2%]
tests/test_info.py::TestMutableByteString::test_frombytes[non-empty] PASSED                                                                                            [  3%]
tests/test_info.py::TestMutableByteString::test_frombytes[empty] PASSED                                                                                                [  4%]
tests/test_info.py::test_info_name_id_5[1-0-no-previous-info-detailed] PASSED                                                                                          [  5%]
tests/test_info.py::test_info_name_id_5[1-0-no-previous-info-no_detailed] PASSED                                                                                       [  6%]
tests/test_info.py::test_info_name_id_5[1-0-previous-info-last-detailed] PASSED                                                                                        [  7%]
tests/test_info.py::test_info_name_id_5[1-0-previous-info-last-no_detailed] PASSED                                                                                     [  8%]
tests/test_info.py::test_info_name_id_5[1-0-previous-info-not-last-detailed] PASSED                                                                                    [  9%]
tests/test_info.py::test_info_name_id_5[1-0-previous-info-not-last-no_detailed] PASSED                                                                                 [ 10%]
tests/test_info.py::test_info_name_id_5[3-1-no-previous-info-detailed] PASSED                                                                                          [ 11%]
tests/test_info.py::test_info_name_id_5[3-1-no-previous-info-no_detailed] PASSED                                                                                       [ 13%]
tests/test_info.py::test_info_name_id_5[3-1-previous-info-last-detailed] PASSED                                                                                        [ 14%]
tests/test_info.py::test_info_name_id_5[3-1-previous-info-last-no_detailed] PASSED                                                                                     [ 15%]
tests/test_info.py::test_info_name_id_5[3-1-previous-info-not-last-detailed] PASSED                                                                                    [ 16%]
tests/test_info.py::test_info_name_id_5[3-1-previous-info-not-last-no_detailed] PASSED                                                                                 [ 17%]
tests/test_info.py::test_info_name_id_5[3-10-no-previous-info-detailed] PASSED                                                                                         [ 18%]
tests/test_info.py::test_info_name_id_5[3-10-no-previous-info-no_detailed] PASSED                                                                                      [ 19%]
tests/test_info.py::test_info_name_id_5[3-10-previous-info-last-detailed] PASSED                                                                                       [ 20%]
tests/test_info.py::test_info_name_id_5[3-10-previous-info-last-no_detailed] PASSED                                                                                    [ 21%]
tests/test_info.py::test_info_name_id_5[3-10-previous-info-not-last-detailed] PASSED                                                                                   [ 22%]
tests/test_info.py::test_info_name_id_5[3-10-previous-info-not-last-no_detailed] PASSED                                                                                [ 23%]
tests/test_info.py::test_info_name_id_5_overflow PASSED                                                                                                                [ 25%]
tests/test_info.py::test_build_info_string_no_detail PASSED                                                                                                            [ 26%]
tests/test_info.py::test_build_info_string_detailed[default] PASSED                                                                                                    [ 27%]
tests/test_info.py::test_build_info_string_detailed[dehint] PASSED                                                                                                     [ 28%]
tests/test_info.py::test_build_info_string_detailed[fallback_stem_width] PASSED                                                                                        [ 29%]
tests/test_info.py::test_build_info_string_detailed[control_name] PASSED                                                                                               [ 30%]
tests/test_info.py::test_build_info_string_detailed[reference_name_and_index] PASSED                                                                                   [ 31%]
tests/test_info.py::test_build_info_string_detailed[gray_stem_width_mode_NATURAL] PASSED                                                                               [ 32%]
tests/test_info.py::test_build_info_string_detailed[gray_stem_width_mode_STRONG] PASSED                                                                                [ 33%]
tests/test_info.py::test_build_info_string_detailed[gdi_cleartype_stem_width_mode_NATURAL] PASSED                                                                      [ 34%]
tests/test_info.py::test_build_info_string_detailed[gdi_cleartype_stem_width_mode_QUANTIZED] PASSED                                                                    [ 35%]
tests/test_info.py::test_build_info_string_detailed[dw_cleartype_stem_width_mode_NATURAL] PASSED                                                                       [ 36%]
tests/test_info.py::test_build_info_string_detailed[dw_cleartype_stem_width_mode_STRONG] PASSED                                                                        [ 38%]
tests/test_info.py::test_build_info_string_detailed[windows_compatibility] PASSED                                                                                      [ 39%]
tests/test_info.py::test_build_info_string_detailed[adjust_subglyphs] PASSED                                                                                           [ 40%]
tests/test_info.py::test_build_info_string_detailed[hint_composites] PASSED                                                                                            [ 41%]
tests/test_info.py::test_build_info_string_detailed[symbol] PASSED                                                                                                     [ 42%]
tests/test_info.py::test_build_info_string_detailed[fallback_scaling] PASSED                                                                                           [ 43%]
tests/test_info.py::test_build_info_string_detailed[TTFA_info] PASSED                                                                                                  [ 44%]
tests/test_info.py::test_build_info_string_detailed[x_height_snapping_exceptions] PASSED                                                                               [ 45%]
tests/test_info.py::test_insert_suffix[is-substring] PASSED                                                                                                            [ 46%]
tests/test_info.py::test_insert_suffix[insert-after-substring] PASSED                                                                                                  [ 47%]
tests/test_info.py::test_insert_suffix[no-substring] PASSED                                                                                                            [ 48%]
tests/test_info.py::test_insert_suffix_overflows PASSED                                                                                                                [ 50%]
tests/test_options.py::TestValidateOptions::test_no_input PASSED                                                                                                       [ 51%]
tests/test_options.py::TestValidateOptions::test_unknown_keyword PASSED                                                                                                [ 52%]
tests/test_options.py::TestValidateOptions::test_no_info_or_detailed_info PASSED                                                                                       [ 53%]
tests/test_options.py::TestValidateOptions::test_in_file_or_in_buffer PASSED                                                                                           [ 54%]
tests/test_options.py::TestValidateOptions::test_control_file_or_control_buffer PASSED                                                                                 [ 55%]
tests/test_options.py::TestValidateOptions::test_reference_file_or_reference_buffer PASSED                                                                             [ 56%]
tests/test_options.py::TestValidateOptions::test_in_file_to_in_buffer PASSED                                                                                           [ 57%]
tests/test_options.py::TestValidateOptions::test_in_buffer_is_bytes PASSED                                                                                             [ 58%]
tests/test_options.py::TestValidateOptions::test_control_file_to_control_buffer PASSED                                                                                 [ 59%]
tests/test_options.py::TestValidateOptions::test_control_buffer_name PASSED                                                                                            [ 60%]
tests/test_options.py::TestValidateOptions::test_reference_file_to_reference_buffer PASSED                                                                             [ 61%]
tests/test_options.py::TestValidateOptions::test_custom_reference_name PASSED                                                                                          [ 63%]
tests/test_options.py::TestValidateOptions::test_reference_buffer_is_bytes PASSED                                                                                      [ 64%]
tests/test_options.py::TestValidateOptions::test_epoch PASSED                                                                                                          [ 65%]
tests/test_options.py::TestValidateOptions::test_family_suffix PASSED                                                                                                  [ 66%]
tests/test_options.py::test_format_varargs[empty] PASSED                                                                                                               [ 67%]
tests/test_options.py::test_format_varargs[full-options] PASSED                                                                                                        [ 68%]
tests/test_options.py::test_format_varargs[unknown-option] PASSED                                                                                                      [ 69%]
tests/test_options.py::test_strong_stem_width[empty-string] PASSED                                                                                                     [ 70%]
tests/test_options.py::test_strong_stem_width[only-gray] PASSED                                                                                                        [ 71%]
tests/test_options.py::test_strong_stem_width[only-gdi] PASSED                                                                                                         [ 72%]
tests/test_options.py::test_strong_stem_width[only-dw] PASSED                                                                                                          [ 73%]
tests/test_options.py::test_strong_stem_width[all] PASSED                                                                                                              [ 75%]
tests/test_options.py::test_strong_stem_width_invalid PASSED                                                                                                           [ 76%]
tests/test_options.py::test_stem_width_mode[nnn] PASSED                                                                                                                [ 77%]
tests/test_options.py::test_stem_width_mode[qqq] PASSED                                                                                                                [ 78%]
tests/test_options.py::test_stem_width_mode[sss] PASSED                                                                                                                [ 79%]
tests/test_options.py::test_stem_width_mode[nqs] PASSED                                                                                                                [ 80%]
tests/test_options.py::test_stem_width_mode_invalid PASSED                                                                                                             [ 81%]
tests/test_options.py::test_stdin_input_type[tty] PASSED                                                                                                               [ 82%]
tests/test_options.py::test_stdin_input_type[pipe] PASSED                                                                                                              [ 83%]
tests/test_options.py::test_path_input_type PASSED                                                                                                                     [ 84%]
tests/test_options.py::test_stdout_output_type[tty] PASSED                                                                                                             [ 85%]
tests/test_options.py::test_stdout_output_type[pipe] PASSED                                                                                                            [ 86%]
tests/test_options.py::test_path_output_type PASSED                                                                                                                    [ 88%]
tests/test_options.py::TestParseArgs::test_unrecognized_arguments PASSED                                                                                               [ 89%]
tests/test_options.py::TestParseArgs::test_no_in_file PASSED                                                                                                           [ 90%]
tests/test_options.py::TestParseArgs::test_no_out_file PASSED                                                                                                          [ 91%]
tests/test_options.py::TestParseArgs::test_source_date_epoch PASSED                                                                                                    [ 92%]
tests/test_options.py::TestParseArgs::test_source_date_epoch_invalid PASSED                                                                                            [ 93%]
tests/test_options.py::TestParseArgs::test_show_ttfa_info_unsupported PASSED                                                                                           [ 94%]
tests/test_ttfautohint.py::TestTTFAutohint::test_simple[NotoSansMono-Regular.ttf] Fatal Python error: Segmentation fault

Current thread 0x0000000104790580 (most recent call first):
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/ttfautohint/__init__.py", line 118 in ttfautohint
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/tests/test_ttfautohint.py", line 21 in autohint_font
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/tests/test_ttfautohint.py", line 42 in test_simple
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/_pytest/python.py", line 183 in pytest_pyfunc_call
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/_pytest/python.py", line 1641 in runtest
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/_pytest/runner.py", line 162 in pytest_runtest_call
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/_pytest/runner.py", line 255 in <lambda>
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/_pytest/runner.py", line 311 in from_call
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/_pytest/runner.py", line 254 in call_runtest_hook
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/_pytest/runner.py", line 215 in call_and_report
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/_pytest/runner.py", line 126 in runtestprotocol
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/_pytest/runner.py", line 109 in pytest_runtest_protocol
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/_pytest/main.py", line 348 in pytest_runtestloop
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/_pytest/main.py", line 323 in _main
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/_pytest/main.py", line 269 in wrap_session
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/_pytest/main.py", line 316 in pytest_cmdline_main
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/pluggy/_callers.py", line 39 in _multicall
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/pluggy/_manager.py", line 80 in _hookexec
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/pluggy/_hooks.py", line 265 in __call__
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 162 in main
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/lib/python3.10/site-packages/_pytest/config/__init__.py", line 185 in console_main
  File "/Users/jeremiehornus/GitHub/ttfautohint-py/venv/bin/pytest", line 8 in <module>
Segmentation fault: 11

[anthrotype]

can you try forcing python to run in x86_64 emulation mode by prepending arch -x86_64 python3 ... to your python invocation? (I think that should work provided your python is 'universal' fat binary containing both x86_64 and arm64 like those from python.org -- homebrew builds from source so you only get arm64) Does the segfault disappear?

[BlackFoundry]

same error with arch -x86_64 python3 …

[anthrotype]

Thanks.. I don't have an M1 mac at hand to debug this, I'm afraid. @simoncozens do you happen to have one and are available to take a look?, or know someone who can?

[simoncozens]

Yeah, I've been getting lots of segfaults myself today.

[simoncozens]

VM Region Info: 0xc800000000 is not in any region.  Bytes after previous region: 377957122049  Bytes before following region: 104694122807296
      REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL
      commpage (reserved)        1000000000-7000000000   [384.0G] ---/--- SM=NUL  ...(unallocated)
--->  GAP OF 0x5f9000000000 BYTES
      MALLOC_NANO              600000000000-600008000000 [128.0M] rw-/rwx SM=PRV  

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   ???                                   0xc800000000 ???
1   libffi.dylib                           0x1c7960050 ffi_call_SYSV + 80
2   libffi.dylib                           0x1c79689e4 ffi_call_int + 948
3   _ctypes.cpython-39-darwin.so           0x10900ceb8 _ctypes_callproc + 864
4   _ctypes.cpython-39-darwin.so           0x109007934 PyCFuncPtr_call + 220
5   Python                                 0x10563d330 _PyObject_Call + 128

[anthrotype]

i think we build the libraries with -g option so you should be able to inspect the core dump in gdb or similar debugger

[simoncozens]

TTF_autohint is jumping into space:

  * frame #0: 0x000000c800000000
    frame #1: 0x0000000107b12dc0 libttfautohint.dylib`TTF_autohint + 380
    frame #2: 0x00000001c7960050 libffi.dylib`ffi_call_SYSV + 80

[anthrotype]

no idea why that would only appen on arm64. Maybe you should try to build from source directly from an arm64 mac, instead of using the cross-compiled wheel built on x64_84, and see if that fixes the issue at all?

[simoncozens]

Giving it a go.

[simoncozens]

Same deal, I'm afraid:

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0xc800000000)
  * frame #0: 0x000000c800000000
    frame #1: 0x000000010755c8e4 libttfautohint.dylib`TTF_autohint(options=<unavailable>) at ttfautohint.c:839:5 [opt]

[simoncozens]

(That's an attempt to call the error callback.)

[simoncozens]

Now I've worked around the error handler segfault, I'm getting an error code 6 with my fonts, which is a freetype error Invalid_Argument.

[anthrotype]

very weird.. it's been long time since i've worked on this codebase (or with ctypes in general) so I have no idea. I finally also have a MBP with M1 Pro chip so I can take a look at some point (more likely Simon fixes it before I do ;)

[simoncozens]

The varargs just aren't going through.

[simoncozens]

varargs just aren't going through in ctypes full stop:

>>> from ctypes import *
>>> from ctypes.util import find_library
>>> libc_path = find_library("c")
>>> libc = cdll.LoadLibrary(libc_path)
>>> libc.printf("%f %f\n", c_float(1.0), c_double(1.0))
0

[anthrotype]

oh dear, have you tried searching the python bugs ?

[simoncozens]

https://bugs.python.org/issue42880

[anthrotype]

does this comment from that linked thread fix/works-around the issue?

AFAIK you should specify the types of the fixed arguments for variadic functions. The ABI for the M1 has a different calling convention of the variadic arguments, the ctypes implementation counts how many variadic arguments there are based on the specified number of fixed arguments.

[simoncozens]

Hooray.

        lib.TTF_autohint.argtypes = [c_char_p]

is indeed the fix.