search

apache / arrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics
https://arrow.apache.org/
Apache License 2.0
14.46k stars 3.52k forks source link

[Python] macOS Segfault on Import, Both arm64 and x86_64 #37010

Open essandess opened 1 year ago

essandess commented 1 year ago

Describe the bug, including details regarding any error messages, version, and platform.

The pyarrow packages create a segfault on import when compiled on macOS boxes. This happens on macOS arm64 and x86_64, using (at least) version 12.0.0 and 13.0.0.

Context: I am trying to update the MacPorts apache-arrow ports at https://github.com/macports/macports-ports/pull/19664 and discovered this issue. MacPorts uses the apache-arrow build instructions and approach in python.rst#build-and-test and python_wheel_macos_build.sh.

Behavior:

$ port installed py310-pyarrow
The following ports are currently installed:
  py310-pyarrow @12.0.0_3 (active)

$ PYTHONFAULTHANDLER=1 python3.10 -c 'import pyarrow'
Fatal Python error: Segmentation fault

Current thread 0x00007ff84d123700 (most recent call first):
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1176 in create_module
  File "<frozen importlib._bootstrap>", line 571 in module_from_spec
  File "<frozen importlib._bootstrap>", line 674 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "/opt/local/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/pyarrow/__init__.py", line 65 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "<string>", line 1 in <module>
Segmentation fault: 11

Component(s)

Python

kou commented 1 year ago

Could you get backtrace by lldb?

essandess commented 1 year ago

Here's the macOS Crash Reporter output. What's the lldb command to capture the trace for python3.10 -c 'import pyarrow'?

Also, I've tried setting a few more cmake flags to be more consistent with the build script at python_wheel_macos_build.sh, but keep hitting the same issue.

Process:               Python [51251]
Path:                  /opt/local/Library/Frameworks/Python.framework/Versions/3.10/Resources/Python.app/Contents/MacOS/Python
Identifier:            org.python.python
Version:               3.10.12 (3.10.12)
Code Type:             X86-64 (Native)
Parent Process:        bash [15799]
Responsible:           Terminal [15797]
User ID:               502

Date/Time:             2023-08-03 16:08:12.4564 -0400
OS Version:            macOS 13.5 (22G74)
Report Version:        12
Bridge OS Version:     7.6 (20P6072)

Time Awake Since Boot: 810000 seconds
Time Since Wake:       607027 seconds

System Integrity Protection: enabled

Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       KERN_INVALID_ADDRESS at 0x0000000000012340
Exception Codes:       0x0000000000000001, 0x0000000000012340

Termination Reason:    Namespace SIGNAL, Code 11 Segmentation fault: 11
Terminating Process:   exc handler [51251]

VM Region Info: 0x12340 is not in any region.  Bytes before following region: 4352785600
      REGION TYPE                    START - END         [ VSIZE] PRT/MAX SHRMOD  REGION DETAIL
      UNUSED SPACE AT START
--->  
      __TEXT                      103736000-10373a000    [   16K] r-x/r-x SM=COW  .../MacOS/Python

Thread 0 Crashed::  Dispatch queue: com.apple.main-thread
0   libjemalloc.2.dylib                    0x1059ec3df je_free_default + 193
1   libprotobuf.3.21.12.0.dylib            0x105688477 google::protobuf::internal::ArenaStringPtr::Destroy() + 39
2   libprotobuf.3.21.12.0.dylib            0x10571d5a5 google::protobuf::FileDescriptorProto::SharedDtor() + 149
3   libprotobuf.3.21.12.0.dylib            0x10571d6bd google::protobuf::FileDescriptorProto::~FileDescriptorProto() + 45
4   libprotobuf.3.21.12.0.dylib            0x105738657 google::protobuf::EncodedDescriptorDatabase::Add(void const*, int) + 167
5   libprotobuf.3.21.12.0.dylib            0x1056e34f4 google::protobuf::DescriptorPool::InternalAddGeneratedFile(void const*, int) + 36
6   libprotobuf.3.21.12.0.dylib            0x1057544d9 google::protobuf::(anonymous namespace)::AddDescriptors(google::protobuf::internal::DescriptorTable const*) + 105
7   dyld                                0x7ff81b4f93fb invocation function for block in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const::$_0::operator()() const + 175
8   dyld                                0x7ff81b537b7a invocation function for block in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 242
9   dyld                                0x7ff81b52bf22 invocation function for block in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 577
10  dyld                                0x7ff81b4dc0af dyld3::MachOFile::forEachLoadCommand(Diagnostics&, void (load_command const*, bool&) block_pointer) const + 245
11  dyld                                0x7ff81b52b0bf dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 175
12  dyld                                0x7ff81b53773a dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 470
13  dyld                                0x7ff81b4f666c dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const + 220
14  dyld                                0x7ff81b4f685a dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 178
…
essandess commented 1 year ago

Here's the lldb trace:

lldb trace ### lldb trace ```sh /opt/local/bin/lldb-mp-16 (lldb) command script import pyarrow PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace. Stack dump: 0. Program arguments: /opt/local/libexec/llvm-16/bin/lldb 1. HandleCommand(command = "command script import pyarrow") Stack dump without symbol names (ensure you have llvm-symbolizer in your PATH or set the environment var `LLVM_SYMBOLIZER_PATH` to point to it): 0 libLLVM.dylib 0x0000000113a91c6c llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) + 56 1 libLLVM.dylib 0x0000000113a90c38 llvm::sys::RunSignalHandlers() + 112 2 libLLVM.dylib 0x0000000113a922f8 SignalHandler(int) + 344 3 libsystem_platform.dylib 0x000000018d386a24 _sigtramp + 56 4 libjemalloc.2.dylib 0x000000010f5940e0 je_free_default + 1104 5 libprotobuf.3.21.12.0.dylib 0x000000010f2471f0 google::protobuf::internal::ArenaStringPtr::Destroy() + 56 6 libprotobuf.3.21.12.0.dylib 0x000000010f2d0000 google::protobuf::FileDescriptorProto::SharedDtor() + 144 7 libprotobuf.3.21.12.0.dylib 0x000000010f2d00e4 google::protobuf::FileDescriptorProto::~FileDescriptorProto() + 48 8 libprotobuf.3.21.12.0.dylib 0x000000010f2ea634 google::protobuf::EncodedDescriptorDatabase::Add(void const*, int) + 168 9 libprotobuf.3.21.12.0.dylib 0x000000010f29e878 google::protobuf::DescriptorPool::InternalAddGeneratedFile(void const*, int) + 40 10 libprotobuf.3.21.12.0.dylib 0x000000010f3056a8 google::protobuf::(anonymous namespace)::AddDescriptors(google::protobuf::internal::DescriptorTable const*) + 136 11 libprotobuf.3.21.12.0.dylib 0x000000010f3056dc google::protobuf::internal::AddDescriptorsRunner::AddDescriptorsRunner(google::protobuf::internal::DescriptorTable const*) + 24 12 dyld 0x000000018d01c1d8 invocation function for block in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const::$_0::operator()() const + 168 13 dyld 0x000000018d05de94 invocation function for block in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 340 14 dyld 0x000000018d0511a4 invocation function for block in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 528 15 dyld 0x000000018cffc2d8 dyld3::MachOFile::forEachLoadCommand(Diagnostics&, void (load_command const*, bool&) block_pointer) const + 296 16 dyld 0x000000018d0501cc dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 192 17 dyld 0x000000018d05d958 dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 516 18 dyld 0x000000018d01885c dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const + 448 19 dyld 0x000000018d018c10 dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array&) const + 220 20 dyld 0x000000018d018bec dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array&) const + 184 21 dyld 0x000000018d018bec dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array&) const + 184 22 dyld 0x000000018d018bec dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array&) const + 184 23 dyld 0x000000018d018bec dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array&) const + 184 24 dyld 0x000000018d018bec dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array&) const + 184 25 dyld 0x000000018d01c264 dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const::$_1::operator()() const + 112 26 dyld 0x000000018d018d90 dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const + 304 27 dyld 0x000000018d036d58 dyld4::APIs::dlopen_from(char const*, int, void*) + 1440 28 Python 0x000000010378ae64 _imp_create_dynamic + 616 29 Python 0x00000001036ce5d0 cfunction_vectorcall_FASTCALL + 80 30 Python 0x0000000103760d94 _PyEval_EvalFrameDefault + 50396 31 Python 0x0000000103763424 _PyEval_Vector + 116 32 Python 0x000000010368859c object_vacall + 224 33 Python 0x0000000103688458 PyObject_CallMethodObjArgs + 92 34 Python 0x00000001037873a8 PyImport_ImportModuleLevelObject + 1224 35 Python 0x000000010375b8f0 _PyEval_EvalFrameDefault + 28728 36 Python 0x0000000103753d74 PyEval_EvalCode + 168 37 Python 0x0000000103750174 builtin_exec + 332 38 Python 0x00000001036ce6cc cfunction_vectorcall_FASTCALL_KEYWORDS + 76 39 Python 0x0000000103760d94 _PyEval_EvalFrameDefault + 50396 40 Python 0x0000000103763424 _PyEval_Vector + 116 41 Python 0x000000010368859c object_vacall + 224 42 Python 0x0000000103688458 PyObject_CallMethodObjArgs + 92 43 Python 0x00000001037873a8 PyImport_ImportModuleLevelObject + 1224 44 Python 0x000000010375b8f0 _PyEval_EvalFrameDefault + 28728 45 Python 0x0000000103753d74 PyEval_EvalCode + 168 46 Python 0x00000001037a5d38 run_eval_code_obj + 84 47 Python 0x00000001037a5c9c run_mod + 112 48 Python 0x00000001037a8084 PyRun_StringFlags + 112 49 liblldb.16.0.6.dylib 0x000000010469effc lldb_private::python::runStringMultiLine(llvm::Twine const&, lldb_private::python::PythonDictionary const&, lldb_private::python::PythonDictionary const&) + 120 50 liblldb.16.0.6.dylib 0x00000001046a5838 lldb_private::ScriptInterpreterPythonImpl::ExecuteMultipleLines(char const*, lldb_private::ExecuteScriptOptions const&) + 960 51 liblldb.16.0.6.dylib 0x00000001046ab314 lldb_private::ScriptInterpreterPythonImpl::LoadScriptingModule(char const*, lldb_private::LoadScriptOptions const&, lldb_private::Status&, std::__1::shared_ptr*, lldb_private::FileSpec) + 2092 52 liblldb.16.0.6.dylib 0x0000000104777158 CommandObjectCommandsScriptImport::DoExecute(lldb_private::Args&, lldb_private::CommandReturnObject&) + 328 53 liblldb.16.0.6.dylib 0x00000001043469c0 lldb_private::CommandObjectParsed::Execute(char const*, lldb_private::CommandReturnObject&) + 484 54 liblldb.16.0.6.dylib 0x000000010433d2e8 lldb_private::CommandInterpreter::HandleCommand(char const*, lldb_private::LazyBool, lldb_private::CommandReturnObject&) + 2136 55 liblldb.16.0.6.dylib 0x00000001043409a0 lldb_private::CommandInterpreter::IOHandlerInputComplete(lldb_private::IOHandler&, std::__1::basic_string, std::__1::allocator>&) + 852 56 liblldb.16.0.6.dylib 0x00000001042731a0 lldb_private::IOHandlerEditline::Run() + 304 57 liblldb.16.0.6.dylib 0x0000000104256968 lldb_private::Debugger::RunIOHandlers() + 140 58 liblldb.16.0.6.dylib 0x0000000104341c14 lldb_private::CommandInterpreter::RunCommandInterpreter(lldb_private::CommandInterpreterRunOptions&) + 156 59 liblldb.16.0.6.dylib 0x00000001040a0614 lldb::SBDebugger::RunCommandInterpreter(bool, bool) + 124 60 lldb 0x0000000102ff48e0 Driver::MainLoop() + 2784 61 lldb 0x0000000102ff541c main + 2168 62 dyld 0x000000018cffff28 start + 2236 LLDB diagnostics will be written to /var/folders/2s/r9z6gsps7gn5wd89c0sqs8k80000gq/T/diagnostics-5f4976 Please include the directory content when filing a bug report Segmentation fault: 11 ```
kou commented 1 year ago

Thanks. It seems that jemalloc is related.

Could you try disabling jemalloc when you build Apache Arrow C++? You can disable jemalloc by specifying the -DARROW_JEMALLOC=OFF CMake option.

essandess commented 1 year ago

Thank you, that fixes this issue. I'll bump this upstream to jemalloc.

kou commented 1 year ago

In general, jemalloc is faster than system memory allocator. So you may want to use jemalloc instead of disabling jemalloc.

Apache Arrow C++ uses bundled jemalloc instead of system jemalloc by default. It means that Apache Arrow C++ and Protobuf use different jemalloc in the same process. It may cause this problem. This problem may be solved by using system jemalloc by specifying -Djemalloc_SOURCE=SYSTEM instead of -DARROW_JEMALLOC=OFF.

essandess commented 1 year ago

Thanks. MacPorts’s apache-arrow already uses these system build flags, so this looks like an issue with jemalloc on macOS.

https://github.com/macports/macports-ports/blob/11afc500de4b609a3ed51465915c8102c8550aac/devel/apache-arrow/Portfile#L156

essandess commented 1 year ago

Several people have pointed out that jemalloc may not be the cause of this issue, so I am reopening the issue here. See:

pitrou commented 1 year ago

Several people have pointed out that jemalloc may not be the cause of this issue, so I am reopening the issue here.

Well, it depends. Does this issue still happen if you disable Arrow's jemalloc integration with -DARROW_JEMALLOC=OFF?

prniii commented 9 months ago

Whether this is the same issue or a different one, I am seeing a crash on 'import pyarrow'. But it is a BAD INSTRUCTION crash. Seeing this on both Apple Silicon M1, and Apple Silicon M1 Max. Big Sur and Ventura using the public wheels downloaded via pip. 12.0.1 is the last version that works, 13.0.0, 14.0.0, 14.0.1 and 14.0.2 all crash with the BAD INSTRUCTION. This is not being seen on a M2 Sonoma machine... Are the wheels being built on a M2 machine picking up a higher optimization or processor instruction flag?

pitrou commented 9 months ago

@prniii Can you get a low-level traceback like in https://github.com/apache/arrow/issues/37010#issuecomment-1664653511 ?

prniii commented 9 months ago

Also this is running under Python.org 's 3.11.5 for macOS.

Is this sufficient?
This is (lldb) bt from within Xcode at the point my app crashed in the call to Py_RunString(). it submitted "import pyarrow"

* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_BAD_INSTRUCTION (code=1, subcode=0x4a03000)
  * frame #0: 0x00000002cb9ae308 libarrow.1400.dylib`_armv7_neon_probe + 72
    frame #1: 0x00000002cb9aeaa4 libarrow.1400.dylib`OPENSSL_cpuid_setup + 924
    frame #2: 0x0000000196a381d8 dyld`invocation function for block in dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const::$_0::operator()() const + 168
    frame #3: 0x0000000196a79c60 dyld`invocation function for block in dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 172
    frame #4: 0x0000000196a6d1a4 dyld`invocation function for block in dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 528
    frame #5: 0x0000000196a182d8 dyld`dyld3::MachOFile::forEachLoadCommand(Diagnostics&, void (load_command const*, bool&) block_pointer) const + 296
    frame #6: 0x0000000196a6c1cc dyld`dyld3::MachOFile::forEachSection(void (dyld3::MachOFile::SectionInfo const&, bool, bool&) block_pointer) const + 192
    frame #7: 0x0000000196a6ecfc dyld`dyld3::MachOFile::forEachInitializerPointerSection(Diagnostics&, void (unsigned int, unsigned int, bool&) block_pointer) const + 160
    frame #8: 0x0000000196a79904 dyld`dyld3::MachOAnalyzer::forEachInitializer(Diagnostics&, dyld3::MachOAnalyzer::VMAddrConverter const&, void (unsigned int) block_pointer, void const*) const + 432
    frame #9: 0x0000000196a3485c dyld`dyld4::Loader::findAndRunAllInitializers(dyld4::RuntimeState&) const + 448
    frame #10: 0x0000000196a34c10 dyld`dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 220
    frame #11: 0x0000000196a34bec dyld`dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 184
    frame #12: 0x0000000196a34bec dyld`dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 184
    frame #13: 0x0000000196a34bec dyld`dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 184
    frame #14: 0x0000000196a34bec dyld`dyld4::Loader::runInitializersBottomUp(dyld4::RuntimeState&, dyld3::Array<dyld4::Loader const*>&) const + 184
    frame #15: 0x0000000196a38264 dyld`dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const::$_1::operator()() const + 112
    frame #16: 0x0000000196a34d90 dyld`dyld4::Loader::runInitializersBottomUpPlusUpwardLinks(dyld4::RuntimeState&) const + 304
    frame #17: 0x0000000196a52d58 dyld`dyld4::APIs::dlopen_from(char const*, int, void*) + 1440
    frame #18: 0x000000012b2004f0 Python`_imp_create_dynamic + 1304
    frame #19: 0x000000012b0fe2d0 Python`cfunction_vectorcall_FASTCALL + 80
    frame #20: 0x000000012b1bf3c8 Python`_PyEval_EvalFrameDefault + 63800
    frame #21: 0x000000012b1c2264 Python`_PyEval_Vector + 156
    frame #22: 0x000000012b09c15c Python`object_vacall + 292
    frame #23: 0x000000012b09bf8c Python`PyObject_CallMethodObjArgs + 92
    frame #24: 0x000000012b1fad10 Python`PyImport_ImportModuleLevelObject + 996
    frame #25: 0x000000012b1a79bc Python`builtin___import__ + 168
    frame #26: 0x000000012b0fe430 Python`cfunction_vectorcall_FASTCALL_KEYWORDS + 80
    frame #27: 0x00000002a3031f68 libshiboken6.abi3.6.6.dylib`feature_import(_object*, _object*, _object*) + 168
    frame #28: 0x000000012b0fda20 Python`cfunction_call + 60
    frame #29: 0x000000012b099738 Python`_PyObject_MakeTpCall + 128
    frame #30: 0x000000012b1b8850 Python`_PyEval_EvalFrameDefault + 36288
    frame #31: 0x000000012b1ae814 Python`PyEval_EvalCode + 276
    frame #32: 0x000000012b1a90a4 Python`builtin_exec + 428
    frame #33: 0x000000012b0fe430 Python`cfunction_vectorcall_FASTCALL_KEYWORDS + 80
    frame #34: 0x000000012b1bf3c8 Python`_PyEval_EvalFrameDefault + 63800
    frame #35: 0x000000012b1c2264 Python`_PyEval_Vector + 156
    frame #36: 0x000000012b09c15c Python`object_vacall + 292
    frame #37: 0x000000012b09bf8c Python`PyObject_CallMethodObjArgs + 92
    frame #38: 0x000000012b1fad10 Python`PyImport_ImportModuleLevelObject + 996
    frame #39: 0x000000012b1a79bc Python`builtin___import__ + 168
    frame #40: 0x000000012b0fe430 Python`cfunction_vectorcall_FASTCALL_KEYWORDS + 80
    frame #41: 0x00000002a3031f68 libshiboken6.abi3.6.6.dylib`feature_import(_object*, _object*, _object*) + 168
    frame #42: 0x000000012b0fda20 Python`cfunction_call + 60
    frame #43: 0x000000012b099738 Python`_PyObject_MakeTpCall + 128
    frame #44: 0x000000012b1b8850 Python`_PyEval_EvalFrameDefault + 36288
    frame #45: 0x000000012b1ae814 Python`PyEval_EvalCode + 276
    frame #46: 0x000000012b22bcbc Python`PyRun_StringFlags + 212
prniii commented 9 months ago

I see that this crash dump may be related to openSSL because I am debugging. Looking at a crash dump file from a Release build, which is an address violation. Will post after I examine the release crash file

pitrou commented 9 months ago

@prniii Oops, it seems like running under lldb may produce the wrong traceback according to https://bugzilla.redhat.com/show_bug.cgi?id=1006474

There is something else that you could try:

  1. enable core dumps (perhaps using ulimit -c unlimited)
  2. run python -c "import pyarrow" (not under lldb)
  3. a core dump should be produced, on which you can run lldb to get a traceback of the actual crash (perhaps using lldb python -c /path/to/core/file)
pitrou commented 9 months ago

Also cc @assignUser

antheus-s commented 1 month ago

Is this issue being investigated?

pitrou commented 1 month ago

@antheus-s If you can reproduce the issue, perhaps you could help us by providing a stack trace of the crash? See https://github.com/apache/arrow/issues/37010#issuecomment-1889644937 for possible instructions.

antheus-s commented 1 month ago

Sadly, I am not knowledgable enough in this library to help, I was just curious if anything was still being investigated, as the last reply was from January. I am using pythonnet, a library built upon Apache Arrow to run C# code in Python and vise versa. It currently causes a runtime exception when I try to import any python library.

pitrou commented 1 month ago

If it happens when you try to import any Python library, then you should report the issue to pythonnet.

antheus-s commented 1 month ago

@pitrou, which I did months ago. However, the issue is caused by Apache Arrow, so I am wondering if anyone is working on a fix.

pitrou commented 1 month ago

How do you know that the issue is caused by Apache Arrow? Can you point us to an analysis of the issue?

antheus-s commented 1 month ago

In my conversations with pythonnet contributors, it was explained to me as an issue that is currently fixed in this repo in this commit with a temporary workaround: https://github.com/apache/arrow/commit/7f0c4070dd723b2f7e1967d7f7f2cccf6fb256b7.

I am looking for progress, as I am currently forced to use a different setup for certain development tasks, but like I said, I am not familiar enough with this repository to dive into details. I'm just asking whether there is any ongoing investigation.

pitrou commented 1 month ago

Thanks for the pointer @antheus-s ! This does not seem related to Arrow since it occurs when importing any Python module IIUC. Also, now that the pythonnet project has a detailed analysis to work with, hopefully they can devise a reliable solution.