search

AnacondaRecipes / arrow-cpp-feedstock

A conda-smithy repository for arrow-cpp.
BSD 3-Clause "New" or "Revised" License
0 stars 3 forks source link

disable mimalloc on osx-arm64 because its initalizer causes a segfault #11

Closed seibert closed 4 months ago

seibert commented 2 years ago

I discovered this segfault when trying to use pyarrow on osx-arm64:

conda create -n arrowtest python=3.9 pyarrow
conda activate arrowtest
python -X faulthandler -c 'import pyarrow'

producing the result:

Fatal Python error: Segmentation fault

Current thread 0x00000001028bc580 (most recent call first):
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1173 in create_module
  File "<frozen importlib._bootstrap>", line 565 in module_from_spec
  File "<frozen importlib._bootstrap>", line 666 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 986 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1007 in _find_and_load
  File "/Users/sseibert/armconda3/envs/arrowtest/lib/python3.9/site-packages/pyarrow/__init__.py", line 63 in <module>
  File "<frozen importlib._bootstrap>", line 228 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 850 in exec_module
  File "<frozen importlib._bootstrap>", line 680 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 986 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1007 in _find_and_load
  File "<string>", line 1 in <module>
Segmentation fault: 11

Using lldb, you can dig into the C stack to see where the segfault happens:

(arrowtest) CONDA-0639-ARM:arrow-cpp-feedstock sseibert$ lldb python
(lldb) target create "python"
Current executable set to 'python' (arm64).
(lldb) run -X faulthandler -c 'import pyarrow'
Process 4761 launched: '/Users/sseibert/armconda3/envs/arrowtest/bin/python' (arm64)
Process 4761 stopped
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x3)
    frame #0: 0x0000000103300f30 libarrow.400.1.0.dylib`_mi_process_init + 28
libarrow.400.1.0.dylib`_mi_process_init:
->  0x103300f30 <+28>: ldr    x8, [x8]
    0x103300f34 <+32>: adr    x19, #0xeb2dc             ; _mi_heap_main
    0x103300f38 <+36>: nop
    0x103300f3c <+40>: str    x8, [x19, #0xb20]
Target 0: (python) stopped.
(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x3)
  * frame #0: 0x0000000103300f30 libarrow.400.1.0.dylib`_mi_process_init + 28
    frame #1: 0x000000010040dc1c dyld`invocation function for block in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const + 164
    frame #2: 0x0000000100436d98 dyld`invocation function for block in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 168
    frame #3: 0x000000010042d988 dyld`invocation function for block in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 532
    frame #4: 0x00000001003f9f98 dyld`dyld3::MachOFile::forEachLoadCommand(Diagnostics&, void (load_command const*, bool&) block_pointer) const + 168
    frame #5: 0x000000010042d72c dyld`dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 192
    frame #6: 0x00000001004366c0 dyld`dyld3::MachOAnalyzer::forEachInitializerPointerSection(Diagnostics&, void (unsigned int, unsigned int, unsigned char const*, bool&) block_pointer) const + 148
    frame #7: 0x00000001004369c0 dyld`dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 432
    frame #8: 0x000000010040db5c dyld`dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const + 172
    frame #9: 0x000000010040dd00 dyld`dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 208
    frame #10: 0x000000010040dcdc dyld`dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 172
    frame #11: 0x000000010040ddcc dyld`dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const + 124
    frame #12: 0x000000010041d734 dyld`dyld4::APIs::dlopen_from(char const*, int, void*) + 512
    frame #13: 0x00000001001c30e8 python`_imp_create_dynamic + 1480
    frame #14: 0x00000001000c5e98 python`cfunction_vectorcall_FASTCALL + 88
    frame #15: 0x0000000100183b30 python`_PyEval_EvalFrameDefault + 2192
    frame #16: 0x0000000100181d48 python`_PyEval_EvalCode + 592
    frame #17: 0x0000000100069c50 python`_PyFunction_Vectorcall + 196
    frame #18: 0x000000010018fbd4 python`call_function + 128
    frame #19: 0x0000000100188d34 python`_PyEval_EvalFrameDefault + 23188
    frame #20: 0x0000000100069d30 python`function_code_fastcall + 168
    frame #21: 0x000000010018fbd4 python`call_function + 128
    frame #22: 0x0000000100187c94 python`_PyEval_EvalFrameDefault + 18932
    frame #23: 0x0000000100069d30 python`function_code_fastcall + 168
    frame #24: 0x000000010018fbd4 python`call_function + 128
    frame #25: 0x0000000100187564 python`_PyEval_EvalFrameDefault + 17092
    frame #26: 0x0000000100069d30 python`function_code_fastcall + 168
    frame #27: 0x000000010018fbd4 python`call_function + 128
    frame #28: 0x0000000100187564 python`_PyEval_EvalFrameDefault + 17092
    frame #29: 0x0000000100069d30 python`function_code_fastcall + 168
    frame #30: 0x000000010018fbd4 python`call_function + 128
    frame #31: 0x0000000100187564 python`_PyEval_EvalFrameDefault + 17092
    frame #32: 0x0000000100069d30 python`function_code_fastcall + 168
    frame #33: 0x000000010006b13c python`object_vacall + 292
    frame #34: 0x000000010006b388 python`_PyObject_CallMethodIdObjArgs + 132
    frame #35: 0x00000001001bdc80 python`PyImport_ImportModuleLevelObject + 1032
    frame #36: 0x00000001001881d0 python`_PyEval_EvalFrameDefault + 20272
    frame #37: 0x0000000100181d48 python`_PyEval_EvalCode + 592
    frame #38: 0x000000010017c1d0 python`builtin_exec + 344
    frame #39: 0x00000001000c5e98 python`cfunction_vectorcall_FASTCALL + 88
    frame #40: 0x0000000100183b30 python`_PyEval_EvalFrameDefault + 2192
    frame #41: 0x0000000100181d48 python`_PyEval_EvalCode + 592
    frame #42: 0x0000000100069c50 python`_PyFunction_Vectorcall + 196
    frame #43: 0x000000010018fbd4 python`call_function + 128
    frame #44: 0x0000000100188d34 python`_PyEval_EvalFrameDefault + 23188
    frame #45: 0x0000000100069d30 python`function_code_fastcall + 168
    frame #46: 0x000000010018fbd4 python`call_function + 128
    frame #47: 0x0000000100187c94 python`_PyEval_EvalFrameDefault + 18932
    frame #48: 0x0000000100069d30 python`function_code_fastcall + 168
    frame #49: 0x000000010018fbd4 python`call_function + 128
    frame #50: 0x0000000100187564 python`_PyEval_EvalFrameDefault + 17092
    frame #51: 0x0000000100069d30 python`function_code_fastcall + 168
    frame #52: 0x000000010018fbd4 python`call_function + 128
    frame #53: 0x0000000100187564 python`_PyEval_EvalFrameDefault + 17092
    frame #54: 0x0000000100069d30 python`function_code_fastcall + 168
    frame #55: 0x000000010006b13c python`object_vacall + 292
    frame #56: 0x000000010006b388 python`_PyObject_CallMethodIdObjArgs + 132
    frame #57: 0x00000001001bdc80 python`PyImport_ImportModuleLevelObject + 1032
    frame #58: 0x00000001001881d0 python`_PyEval_EvalFrameDefault + 20272
    frame #59: 0x0000000100181d48 python`_PyEval_EvalCode + 592
    frame #60: 0x00000001001e927c python`PyRun_StringFlags + 308
    frame #61: 0x00000001001e9084 python`PyRun_SimpleStringFlags + 84
    frame #62: 0x000000010020c29c python`pymain_run_command + 148
    frame #63: 0x000000010020b1b4 python`pymain_run_python + 296
    frame #64: 0x000000010020b034 python`Py_RunMain + 40
    frame #65: 0x000000010020c9b4 python`pymain_main + 72
    frame #66: 0x0000000100007b28 python`main + 56
    frame #67: 0x00000001003fd0f4 dyld`start + 520

The _mi_process_init function is part of mimalloc. The most straightforward fix is to disable mimalloc in the build process, which is what this PR does.

The URL for the Arrow source no longer works (apparently old versions are moved to archive.apache.org), so this PR also fixes that.