scikit-hep / uproot5

ROOT I/O in pure Python and NumPy.
https://uproot.readthedocs.io
BSD 3-Clause "New" or "Revised" License
239 stars 76 forks source link

test_0927_dont_assume_uproot_in_global_scope_in_TPython_Eval now raises an exception #1237

Open jpivarski opened 5 months ago

jpivarski commented 5 months ago

It was introduced in #927 to fix a segfault, and now it's producing segfaults again, but probably for a different reason. pytest hides the ROOT error messages, but they are:

 *** Break *** segmentation violation
 Generating stack trace...
 0x00005d42557bf44a in PyUnicode_FromFormatV + 0x88a from python
 0x00005d42557edc81 in PyErr_Format + 0xf1 from python
 0x00005d42556d6851 in <unknown> from python
 0x00005d42557c7c9c in _PyEval_EvalFrameDefault + 0xffc from python
 0x00005d425587ea8d in <unknown> from python
 0x00005d425587e11f in PyEval_EvalCode + 0x9f from python
 0x00005d425589d08a in <unknown> from python
 0x00005d4255898c13 in <unknown> from python
 0x00005d425588d3c2 in PyRun_StringFlags + 0x62 from python
 0x0000724653f49dcd in TPython::Exec(char const*) + 0x3d from /home/jpivarski/miniforge3/lib/libROOTTPython.so.6.32.00
 0x0000724653f5609b in <unknown function>
 0x0000724665957266 in TBuffer::Expand(int, bool) + 0x76 from /home/jpivarski/miniforge3/lib/python3.11/lib-dynload/../.././libCore.so.6.32
 0x000072466526403e in TBufferFile::WriteFastArray(double const*, long long) + 0xbe from /home/jpivarski/miniforge3/lib/libRIO.so
 0x0000724665a9496b in TStreamerBase::WriteBuffer(TBuffer&, char*) + 0x2b from /home/jpivarski/miniforge3/lib/python3.11/lib-dynload/../.././libCore.so.6.32
 0x00007246654ffb84 in int TStreamerInfo::WriteBufferAux<char**>(TBuffer&, char** const&, TStreamerInfo::TCompInfo* const*, int, int, int, int, int) + 0x6504 from /home/jpivarski/miniforge3/lib/libRIO.so
 0x00007246653477e1 in TStreamerInfoActions::GenericWriteAction(TBuffer&, void*, TStreamerInfoActions::TConfiguration const*) + 0x51 from /home/jpivarski/miniforge3/lib/libRIO.so
 0x000072466526bf9e in TBufferFile::WriteClassBuffer(TClass const*, void*) + 0x2ae from /home/jpivarski/miniforge3/lib/libRIO.so
 0x0000724665a9496b in TStreamerBase::WriteBuffer(TBuffer&, char*) + 0x2b from /home/jpivarski/miniforge3/lib/python3.11/lib-dynload/../.././libCore.so.6.32
 0x00007246654ffb84 in int TStreamerInfo::WriteBufferAux<char**>(TBuffer&, char** const&, TStreamerInfo::TCompInfo* const*, int, int, int, int, int) + 0x6504 from /home/jpivarski/miniforge3/lib/libRIO.so
 0x00007246653477e1 in TStreamerInfoActions::GenericWriteAction(TBuffer&, void*, TStreamerInfoActions::TConfiguration const*) + 0x51 from /home/jpivarski/miniforge3/lib/libRIO.so
 0x000072466526bf9e in TBufferFile::WriteClassBuffer(TClass const*, void*) + 0x2ae from /home/jpivarski/miniforge3/lib/libRIO.so
 0x000072466526a69b in TBufferFile::WriteObjectClass(void const*, TClass const*, bool) + 0x1ab from /home/jpivarski/miniforge3/lib/libRIO.so
 0x0000724665d32bfe in <unknown> from /home/jpivarski/miniforge3/lib/python3.11/lib-dynload/../../libcppyy_backend.so
 0x0000724644f7a356 in <unknown> from /home/jpivarski/miniforge3/lib/python3.11/site-packages/libcppyy.so
 0x0000724644f6061a in <unknown> from /home/jpivarski/miniforge3/lib/python3.11/site-packages/libcppyy.so
 0x0000724644f607c8 in <unknown> from /home/jpivarski/miniforge3/lib/python3.11/site-packages/libcppyy.so
 0x0000724644f60956 in CPyCppyy::CPPMethod::Execute(void*, long, CPyCppyy::CallContext*) + 0x26 from /home/jpivarski/miniforge3/lib/python3.11/site-packages/libcppyy.so
 0x0000724644f60ac6 in CPyCppyy::CPPMethod::Call(CPyCppyy::CPPInstance*&, _object* const*, unsigned long, _object*, CPyCppyy::CallContext*) + 0x116 from /home/jpivarski/miniforge3/lib/python3.11/site-packages/libcppyy.so
 0x0000724644f689d3 in <unknown> from /home/jpivarski/miniforge3/lib/python3.11/site-packages/libcppyy.so
 0x00005d42557d3eac in PyObject_Vectorcall + 0x2c from python
 0x00005d42557c73b6 in _PyEval_EvalFrameDefault + 0x716 from python
 0x00005d425587ea8d in <unknown> from python
 0x00005d425587e11f in PyEval_EvalCode + 0x9f from python
 0x00005d425589d08a in <unknown> from python
 0x00005d4255898c13 in <unknown> from python
 0x00005d425588d3c2 in PyRun_StringFlags + 0x62 from python
 0x00005d425588d17c in PyRun_SimpleStringFlags + 0x3c from python
 0x00005d42558a7bdf in Py_RunMain + 0x23f from python
 0x00005d425586e027 in Py_BytesMain + 0x37 from python
 0x0000724667c29d90 in <unknown> from /lib/x86_64-linux-gnu/libc.so.6
 0x0000724667c29e40 in __libc_start_main + 0x80 from /lib/x86_64-linux-gnu/libc.so.6
 0x00005d425586decd in <unknown> from python
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/jpivarski/irishep/uproot5/tests/test_0927_dont_assume_uproot_in_global_scope_in_TPython_Eval.py", line 14, in test
    assert __import__("uproot").from_pyroot(h).values().shape == (1, 1, 1)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jpivarski/irishep/uproot5/src/uproot/pyroot.py", line 258, in from_pyroot
    buffer = pyroot_to_buffer(obj)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jpivarski/irishep/uproot5/src/uproot/pyroot.py", line 136, in pyroot_to_buffer
    message.WriteObject(obj)
cppyy.ll.SegmentationViolation: void TBufferIO::WriteObject(const TObject* obj, Bool_t cacheReuse = kTRUE) =>
    SegmentationViolation: segfault in C++; program state was reset
 *** Break *** segmentation violation
 Generating stack trace...
 0x00005d42557aef74 in <unknown> from python
 0x00005d42557ae513 in <unknown> from python
 0x00005d425587a6fb in <unknown> from python
 0x00005d42558ac481 in PyGC_Collect + 0x61 from python
 0x00005d425589c01e in Py_FinalizeEx + 0x13e from python
 0x00005d42558a7b20 in Py_RunMain + 0x180 from python
 0x00005d425586e027 in Py_BytesMain + 0x37 from python
 0x0000724667c29d90 in <unknown> from /lib/x86_64-linux-gnu/libc.so.6
 0x0000724667c29e40 in __libc_start_main + 0x80 from /lib/x86_64-linux-gnu/libc.so.6
 0x00005d425586decd in <unknown> from python
 *** Break *** segmentation violation
 Generating stack trace...
 0x00005d42557aef74 in <unknown> from python
 0x00005d42557ae513 in <unknown> from python
 0x00005d425587a6fb in <unknown> from python
 0x00005d42558ac481 in PyGC_Collect + 0x61 from python
 0x00005d425589c01e in Py_FinalizeEx + 0x13e from python
 0x00005d42558a7b20 in Py_RunMain + 0x180 from python
 0x00005d425586e027 in Py_BytesMain + 0x37 from python
 0x0000724667c29d90 in <unknown> from /lib/x86_64-linux-gnu/libc.so.6
 0x0000724667c29e40 in __libc_start_main + 0x80 from /lib/x86_64-linux-gnu/libc.so.6
 0x00005d425586decd in <unknown> from python

The segfault is somewhere in the TBufferIO::WriteObject call.

This is a blocker: our one test that imports ROOT always fails because of this.

jpivarski commented 5 months ago

Both tests/test_0927_dont_assume_uproot_in_global_scope_in_TPython_Eval.py (above) and tests/test_0965_inverted_axes_variances_hist_888.py::test_axes_variances_to_hist_3D_weighted have segfaults, and both of them involve conversions between PyROOT and Uproot objects (probably the same error).

Here's the ROOT error output for the second one:

 *** Break *** segmentation violation
 Generating stack trace...
 0x0000646d3a91244a in PyUnicode_FromFormatV + 0x88a from python
 0x0000646d3a940c81 in PyErr_Format + 0xf1 from python
 0x0000646d3a829851 in <unknown> from python
 0x0000646d3a91ac9c in _PyEval_EvalFrameDefault + 0xffc from python
 0x0000646d3a9d1a8d in <unknown> from python
 0x0000646d3a9d111f in PyEval_EvalCode + 0x9f from python
 0x0000646d3a9f008a in <unknown> from python
 0x0000646d3a9ebc13 in <unknown> from python
 0x0000646d3a9e03c2 in PyRun_StringFlags + 0x62 from python
 0x00007c9865a1adcd in TPython::Exec(char const*) + 0x3d from /home/jpivarski/miniforge3/lib/libROOTTPython.so.6.32.00
 0x00007c9865a2709b in <unknown function>
 0x00007c986fb57266 in TBuffer::Expand(int, bool) + 0x76 from /home/jpivarski/miniforge3/lib/python3.11/lib-dynload/../.././libCore.so.6.32
 0x00007c986f66403e in TBufferFile::WriteFastArray(double const*, long long) + 0xbe from /home/jpivarski/miniforge3/lib/libRIO.so
 0x00007c986f665379 in TBufferFile::WriteFastArray(void*, TClass const*, long long, TMemberStreamer*) + 0x99 from /home/jpivarski/miniforge3/lib/libRIO.so
 0x00007c986f8fd220 in int TStreamerInfo::WriteBufferAux<char**>(TBuffer&, char** const&, TStreamerInfo::TCompInfo* const*, int, int, int, int, int) + 0x3ba0 from /home/jpivarski/miniforge3/lib/libRIO.so
 0x00007c986f7477e1 in TStreamerInfoActions::GenericWriteAction(TBuffer&, void*, TStreamerInfoActions::TConfiguration const*) + 0x51 from /home/jpivarski/miniforge3/lib/libRIO.so
 0x00007c986f66bf9e in TBufferFile::WriteClassBuffer(TClass const*, void*) + 0x2ae from /home/jpivarski/miniforge3/lib/libRIO.so
 0x00007c986fc9496b in TStreamerBase::WriteBuffer(TBuffer&, char*) + 0x2b from /home/jpivarski/miniforge3/lib/python3.11/lib-dynload/../.././libCore.so.6.32
 0x00007c986f8ffb84 in int TStreamerInfo::WriteBufferAux<char**>(TBuffer&, char** const&, TStreamerInfo::TCompInfo* const*, int, int, int, int, int) + 0x6504 from /home/jpivarski/miniforge3/lib/libRIO.so
 0x00007c986f7477e1 in TStreamerInfoActions::GenericWriteAction(TBuffer&, void*, TStreamerInfoActions::TConfiguration const*) + 0x51 from /home/jpivarski/miniforge3/lib/libRIO.so
 0x00007c986f66bf9e in TBufferFile::WriteClassBuffer(TClass const*, void*) + 0x2ae from /home/jpivarski/miniforge3/lib/libRIO.so
 0x00007c986fc9496b in TStreamerBase::WriteBuffer(TBuffer&, char*) + 0x2b from /home/jpivarski/miniforge3/lib/python3.11/lib-dynload/../.././libCore.so.6.32
 0x00007c986f8ffb84 in int TStreamerInfo::WriteBufferAux<char**>(TBuffer&, char** const&, TStreamerInfo::TCompInfo* const*, int, int, int, int, int) + 0x6504 from /home/jpivarski/miniforge3/lib/libRIO.so
 0x00007c986f7477e1 in TStreamerInfoActions::GenericWriteAction(TBuffer&, void*, TStreamerInfoActions::TConfiguration const*) + 0x51 from /home/jpivarski/miniforge3/lib/libRIO.so
 0x00007c986f66bf9e in TBufferFile::WriteClassBuffer(TClass const*, void*) + 0x2ae from /home/jpivarski/miniforge3/lib/libRIO.so
 0x00007c986f66a69b in TBufferFile::WriteObjectClass(void const*, TClass const*, bool) + 0x1ab from /home/jpivarski/miniforge3/lib/libRIO.so
 0x00007c98f26b8bfe in <unknown> from /home/jpivarski/miniforge3/lib/python3.11/lib-dynload/../../libcppyy_backend.so
 0x00007c984fbec356 in <unknown> from /home/jpivarski/miniforge3/lib/python3.11/site-packages/libcppyy.so
 0x00007c984fbd261a in <unknown> from /home/jpivarski/miniforge3/lib/python3.11/site-packages/libcppyy.so
 0x00007c984fbd27c8 in <unknown> from /home/jpivarski/miniforge3/lib/python3.11/site-packages/libcppyy.so
 0x00007c984fbd2956 in CPyCppyy::CPPMethod::Execute(void*, long, CPyCppyy::CallContext*) + 0x26 from /home/jpivarski/miniforge3/lib/python3.11/site-packages/libcppyy.so
 0x00007c984fbd2ac6 in CPyCppyy::CPPMethod::Call(CPyCppyy::CPPInstance*&, _object* const*, unsigned long, _object*, CPyCppyy::CallContext*) + 0x116 from /home/jpivarski/miniforge3/lib/python3.11/site-packages/libcppyy.so
 0x00007c984fbda9d3 in <unknown> from /home/jpivarski/miniforge3/lib/python3.11/site-packages/libcppyy.so
 0x0000646d3a926eac in PyObject_Vectorcall + 0x2c from python
 0x0000646d3a91a3b6 in _PyEval_EvalFrameDefault + 0x716 from python
 0x0000646d3a9d1a8d in <unknown> from python
 0x0000646d3a9d111f in PyEval_EvalCode + 0x9f from python
 0x0000646d3a9f008a in <unknown> from python
 0x0000646d3a9ebc13 in <unknown> from python
 0x0000646d3a9e03c2 in PyRun_StringFlags + 0x62 from python
 0x0000646d3a9e017c in PyRun_SimpleStringFlags + 0x3c from python
 0x0000646d3a9fabdf in Py_RunMain + 0x23f from python
 0x0000646d3a9c1027 in Py_BytesMain + 0x37 from python
 0x00007c9901629d90 in <unknown> from /lib/x86_64-linux-gnu/libc.so.6
 0x00007c9901629e40 in __libc_start_main + 0x80 from /lib/x86_64-linux-gnu/libc.so.6
 0x0000646d3a9c0ecd in <unknown> from python
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/jpivarski/irishep/uproot5/tests/test_0965_inverted_axes_variances_hist_888.py", line 44, in test_axes_variances_to_hist_3D_weighted
    huproot3 = uproot.from_pyroot(hroot3)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jpivarski/irishep/uproot5/src/uproot/pyroot.py", line 258, in from_pyroot
    buffer = pyroot_to_buffer(obj)
             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jpivarski/irishep/uproot5/src/uproot/pyroot.py", line 136, in pyroot_to_buffer
    message.WriteObject(obj)
cppyy.ll.SegmentationViolation: void TBufferIO::WriteObject(const TObject* obj, Bool_t cacheReuse = kTRUE) =>
    SegmentationViolation: segfault in C++; program state was reset
 *** Break *** segmentation violation
 Generating stack trace...
 0x0000646d3a901f74 in <unknown> from python
 0x0000646d3a901513 in <unknown> from python
 0x0000646d3a9cd6fb in <unknown> from python
 0x0000646d3a9ff481 in PyGC_Collect + 0x61 from python
 0x0000646d3a9ef01e in Py_FinalizeEx + 0x13e from python
 0x0000646d3a9fab20 in Py_RunMain + 0x180 from python
 0x0000646d3a9c1027 in Py_BytesMain + 0x37 from python
 0x00007c9901629d90 in <unknown> from /lib/x86_64-linux-gnu/libc.so.6
 0x00007c9901629e40 in __libc_start_main + 0x80 from /lib/x86_64-linux-gnu/libc.so.6
 0x0000646d3a9c0ecd in <unknown> from python
 *** Break *** segmentation violation
 Generating stack trace...
 0x0000646d3a901f74 in <unknown> from python
 0x0000646d3a901513 in <unknown> from python
 0x0000646d3a9cd6fb in <unknown> from python
 0x0000646d3a9ff481 in PyGC_Collect + 0x61 from python
 0x0000646d3a9ef01e in Py_FinalizeEx + 0x13e from python
 0x0000646d3a9fab20 in Py_RunMain + 0x180 from python
 0x0000646d3a9c1027 in Py_BytesMain + 0x37 from python
 0x00007c9901629d90 in <unknown> from /lib/x86_64-linux-gnu/libc.so.6
 0x00007c9901629e40 in __libc_start_main + 0x80 from /lib/x86_64-linux-gnu/libc.so.6
 0x0000646d3a9c0ecd in <unknown> from python