search

hibtc / cpymad

Cython binding to MAD-X
http://hibtc.github.io/cpymad/
Other
27 stars 18 forks source link

crashes on windows py35+ (win10 64bit only?) #41

Closed coldfix closed 5 years ago

coldfix commented 6 years ago

Continuing from #32:

We now have py35 and py36 builds of cpymad produced by manual compilation using mingw from WinPython3.4Qt5 (for a more detailed build description, see below).

While building the C extensions works fine, I found that the MAD-X background process crashes in a significant number of cases (between 10% and 90% depending on my PC's mood). At first, I thought this is a problem due to the new build process and the incompatibility between the different CRTs (ucrt in python3.5+ on the on hand, msvcr100 in winpy34 on the other side), but then I discovered that this problem seems to occur only on 64bit on win10. So far, I neither encountered problems on win7 64bit (using the exact same binary) nor on win7/10 32bit (using the same build process).

Some references on the CRT topic are given in https://github.com/hibtc/cpymad/issues/32#issuecomment-153695711.

Traceback

While debugging, I found that this happens during startup when invoking cpymad.libmadx.start(). Attaching a traceback obtained by running gdb on the following mx.py:

import cpymad.libmadx as l
l.start()

Traceback:

C:\Users\thomas\hit\MAD-X-5.03.07\install\bin>gdb python
GNU gdb (GDB) 7.1.90.20100730-cvs
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-w64-mingw32".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from C:\Users\thomas\hit\python\WinPython-64bit-3.6.3.0\python-3.6.3.amd64/python.exe...(no debugging symbols found)...done.
(gdb) run mx.py
Starting program: C:\Users\thomas\hit\python\WinPython-64bit-3.6.3.0\python-3.6.3.amd64/python.exe mx.py
[New Thread 1968.0x1c54]
[New Thread 1968.0x13f4]
[New Thread 1968.0xb18]
[New Thread 1968.0xfc]

Program received signal SIGSEGV, Segmentation fault.
0x000000006823e2c8 in add_to_var_list ()
   from C:\Users\thomas\hit\MAD-X-5.03.07\install\bin\libmadx.dll
(gdb) bt
#0  0x000000006823e2c8 in add_to_var_list ()
   from C:\Users\thomas\hit\MAD-X-5.03.07\install\bin\libmadx.dll
#1  0x000000006823eced in enter_variable ()
   from C:\Users\thomas\hit\MAD-X-5.03.07\install\bin\libmadx.dll
#2  0x00000000681eec55 in process () from C:\Users\thomas\hit\MAD-X-5.03.07\install\bin\libmadx.dll
#3  0x00000000681f0b74 in pro_input_ ()
   from C:\Users\thomas\hit\MAD-X-5.03.07\install\bin\libmadx.dll
#4  0x000000006823d720 in get_defined_constants ()
   from C:\Users\thomas\hit\MAD-X-5.03.07\install\bin\libmadx.dll
#5  0x00000000681e62c7 in madx_start ()
   from C:\Users\thomas\hit\MAD-X-5.03.07\install\bin\libmadx.dll
#6  0x00000000735307a3 in _ctypes!DllCanUnloadNow ()
   from C:\Users\thomas\hit\python\WinPython-64bit-3.6.3.0\python-3.6.3.amd64\DLLs\_ctypes.pyd
#7  0x00000056497eefe0 in ?? ()
#8  0x000000007353075a in _ctypes!DllCanUnloadNow ()
   from C:\Users\thomas\hit\python\WinPython-64bit-3.6.3.0\python-3.6.3.amd64\DLLs\_ctypes.pyd
warning: (Internal error: pc 0x2 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x2 in read in psymtab, but not in symtab.)

#9  0x0000000000000003 in ?? (warning: (Internal error: pc 0x2 in read in psymtab, but not in symtab.)

)
    at C:/Users/thomas/hit/python/WinPython-64bit-3.4.4.4/python-3.4.4.amd64/share/mingwpy/x86_64-w64-mingw32/include/c++/bits/stl_vector.h:423
warning: (Internal error: pc 0x2 in read in psymtab, but not in symtab.)

#10 0x0000000073568ba3 in python36!_PyEval_EvalFrameDefault ()
   from C:\Users\thomas\hit\python\WinPython-64bit-3.6.3.0\python-3.6.3.amd64\python36.dll
#11 0x00000000738394d0 in python36!_PyNamespace_Type ()
   from C:\Users\thomas\hit\python\WinPython-64bit-3.6.3.0\python-3.6.3.amd64\python36.dll
#12 0x000001caddc0e340 in ?? ()
#13 0x00000056497eee10 in ?? ()
#14 0x0000000073589cb0 in python36!PyUnicode_Partition ()
   from C:\Users\thomas\hit\python\WinPython-64bit-3.6.3.0\python-3.6.3.amd64\python36.dll
#15 0x00000056497eee90 in ?? ()
#16 0x000000007352f0b3 in _ctypes!DllCanUnloadNow ()
   from C:\Users\thomas\hit\python\WinPython-64bit-3.6.3.0\python-3.6.3.amd64\DLLs\_ctypes.pyd
#17 0x000000007352ed90 in _ctypes!DllCanUnloadNow ()
   from C:\Users\thomas\hit\python\WinPython-64bit-3.6.3.0\python-3.6.3.amd64\DLLs\_ctypes.pyd
#18 0x00000056497eee90 in ?? ()
warning: (Internal error: pc 0x27 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x27 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x27 in read in psymtab, but not in symtab.)

#19 0x0000000000000028 in ?? (warning: (Internal error: pc 0x27 in read in psymtab, but not in symtab.)

)
    at C:/Users/thomas/hit/python/WinPython-64bit-3.4.4.4/python-3.4.4.amd64/share/mingwpy/x86_64-w64-mingw32/include/c++/bits/stl_construct.h:102
warning: (Internal error: pc 0x27 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x0 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x0 in read in psymtab, but not in symtab.)

warning: (Internal error: pc 0x0 in read in psymtab, but not in symtab.)

#20 0x0000000000000001 in ?? (warning: (Internal error: pc 0x0 in read in psymtab, but not in symtab.)

)
    at C:/Users/thomas/hit/python/WinPython-64bit-3.4.4.4/python-3.4.4.amd64/share/mingwpy/x86_64-w64-mingw32/include/c++/bits/stl_vector.h:423
warning: (Internal error: pc 0x0 in read in psymtab, but not in symtab.)

#21 0x00000056497ef070 in ?? ()
#22 0x00000000681e5ed0 in update_sequ_constraints ()
   from C:\Users\thomas\hit\MAD-X-5.03.07\install\bin\libmadx.dll
#23 0x00000056497eefe0 in ?? ()
#24 0x00000056497ef070 in ?? ()
#25 0x00000056497ef070 in ?? ()
#26 0x0000d93af7e47872 in ?? ()
#27 0x000001caddc5cb60 in ?? ()
#28 0x000001cadffcb240 in ?? ()
#29 0x0000000000000000 in ?? ()

Build description

First build MAD-X 5.04.00 as a static library (once on 32bit plus once on 64bit):

mkdir MAD-X-5.04.00\build
cd MAD-X-5.04.00\build
cmake .. -G "MinGW Makefiles" ^
    -DCMAKE_INSTALL_PREFIX=..\installStatic ^
    -DBUILD_SHARED_LIBS=OFF ^
    -DMADX_STATIC=ON ^
    -DUSE_GC=ON ^
    -DMADX_ONLINE=OFF ^
    -DCMAKE_BUILD_TYPE=Release
mingw32-make install

Then build cpymad. Here assuming py35/64bit:

set pythondir=..\WinPython-64bit-3.5.4.1\python-3.5.4.amd64
set madxdir=..\MAD-X-5.04.00\installStatic
set gcc=..\WinPython-64bit-3.4.4.4\python-3.4.4.amd64\Scripts\gcc.exe
set platform=win-amd64-3.5
set pyd_code=cp35-win_amd64
set pymajor=35
set tempdir=build\temp.%platform%\Release\cpymad
set builddir=build\lib.%platform%\cpymad

cython cpymad\libmadx.pyx -I %madxdir%\include
python setup.py build_py

%gcc% -mdll -O -Wall ^
    -I%madxdir%\include ^
    -I%pythondir%\include ^
    -c cpymad/libmadx.c ^
    -o %tempdir%\libmadx.obj ^
    -std=gnu99

%gcc -shared -s ^
    %tempdir%\libmadx.obj ^
    -L%madxdir%\lib ^
    -L%pythondir%\libs
    -lmadx -lptc -lgc-lib -lstdc++ -lgfortran -lquadmath -lpython%pymajor% -lmsvcr100 ^
    -o %builddir%\libmadx.%pyd_code%.pyd

python setup.py bdist_wheel
coldfix commented 5 years ago

Finally some good news on this. It seems that the crash is caused by the garbage collector (bdwgc) and can be solved by upgrading to a newer version. No crashs found with 7.6.8, I will do a post-release soon. Please reopen if similar issues happen on a cpymad version greater than 1.0.10.