search

bastibe / transplant

Transplant is an easy way of calling Matlab from Python
https://transplant.readthedocs.io
Other
110 stars 26 forks source link

Windows, Matlab process dies on start #60

Closed zanieb closed 6 years ago

zanieb commented 6 years ago

When opening a Matlab instance on Windows, the Matlab process dies immediately. Transplant throws a runtime error. No error message is displayed in the Matlab command window, it becomes unresponsive.

Attempted solutions

Reinstall all relevant packages

No change

Running with admin permissions

No change

Install ZMQ without Anaconda

Same problem as far as I can tell

Fully removed:

In [2]: transplant.Matlab()
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-2-d12f35e2c1f4> in <module>()
----> 1 transplant.Matlab()

C:\ProgramData\Anaconda3\lib\site-packages\transplant\transplant_master.py in __
init__(self, executable, arguments, msgformat, address, user, print_to_stdout, d
esktop, jvm)
    468                                   "transplant_remote('{}','{}','{}');".f
ormat(
    469                                       os.path.dirname(__file__), os.getc
wd(),
--> 470                                       msgformat, zmq_address, self._loca
te_libzmq()
    471 )])
    472         else:

C:\ProgramData\Anaconda3\lib\site-packages\transplant\transplant_master.py in _l
ocate_libzmq(self)
    606                 return candidates[0]
    607
--> 608         raise RuntimeError('could not locate libzmq for Matlab')
    609
    610     def _ask_ld_for_paths(self):

RuntimeError: could not locate libzmq for Matlab

Reinstalled with ZeroMQ executable:

In [2]: transplant.Matlab()
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-2-d12f35e2c1f4> in <module>()
----> 1 transplant.Matlab()
...
RuntimeError: Process died unexpectedly

In [3]: %debug
> c:\programdata\anaconda3\lib\site-packages\transplant\transplant_master.py(178
)_wait_socket()
    176         while True:
    177             if self.process.poll() is not None:
--> 178                 raise RuntimeError('Process died unexpectedly')
    179             if self.socket.poll(timeout, flags) != 0:
    180                 return

ipdb> self._locate_libzmq()
'C:/Program Files\\ZeroMQ 4.0.4\\bin\\libzmq-v100-mt-4_0_4.dll'

Load the ZMQ library from Matlab

No errors e.g.

loadlibrary('C:\ProgramData\Anaconda3\Library\bin\libzmq.dll', 'C:\ProgramData\Anaconda3\pkgs\zeromq-4.2.3-hd6b2f15_3\Library\include\zmq.h')

Create a ZMQ (transplant) class instance from within Matlab

Using the path given by _locate_libzmq() Can create the ZMQ object without problems

e.g.

ZMQ('C:\ProgramData\Anaconda3\Library\bin\libzmq.dll', 'tcp://127.0.0.1:65535')

Install on another (fresh) Windows machine

Same problem

Installation steps

conda create --name test
activate test
conda install zeromq
pip install transplant

(Visual C++ was already installed)

Use Visual C++ 2015 instead of MinGW/GCC

As a Matlab compiler

Reproduction and error messages

Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.
C:\Windows\system32>ipython
Python 3.5.5 |Anaconda custom (64-bit)| (default, Apr  7 2018, 04:52:34) [MSC v.
1900 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 6.3.1 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import transplant

In [2]: transplant.Matlab()
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-2-d12f35e2c1f4> in <module>()
----> 1 transplant.Matlab()

C:\ProgramData\Anaconda3\lib\site-packages\transplant\transplant_master.py in __
init__(self, executable, arguments, msgformat, address, user, print_to_stdout, d
esktop, jvm)
    495         if print_to_stdout:
    496             self._start_reader()
--> 497         self.eval('0;') # no-op. Wait for Matlab startup to complete.
    498
    499     def exit(self):

C:\ProgramData\Anaconda3\lib\site-packages\transplant\transplant_master.py in __
getattr__(self, name)
    532         """Retrieve a value or function from the remote."""
    533         try:
--> 534             return self._get_global(name)
    535         except TransplantError as err:
    536             # package identifiers for `what` use '/' instead of '.':

C:\ProgramData\Anaconda3\lib\site-packages\transplant\transplant_master.py in _g
et_global(self, name)
     90     def _get_global(self, name):
     91         """Retrieve a value from a named variable."""
---> 92         response = self.send_message('get_global', name=name)
     93         return response['value']
     94

C:\ProgramData\Anaconda3\lib\site-packages\transplant\transplant_master.py in se
nd_message(self, msg_type, **kwargs)
    146         kwargs = self._encode_values(kwargs)
    147
--> 148         self._wait_socket(zmq.POLLOUT)
    149         if self.msgformat == 'msgpack':
    150             self.socket.send(msgpack.packb(dict(kwargs, type=msg_type),
use_bin_type=True), flags=zmq.NOBLOCK)

C:\ProgramData\Anaconda3\lib\site-packages\transplant\transplant_master.py in _w
ait_socket(self, flags, timeout)
    176         while True:
    177             if self.process.poll() is not None:
--> 178                 raise RuntimeError('Process died unexpectedly')
    179             if self.socket.poll(timeout, flags) != 0:
    180                 return

RuntimeError: Process died unexpectedly

Matlab crash dump

------------------------------------------------------------------------
    Unknown exception 0x40000015 detected at Tue Apr 17 14:05:36 2018
------------------------------------------------------------------------

Configuration:
  Crash Decoding      : Disabled - No sandbox or build area path
  Crash Mode          : continue (default)
  Current Graphics Driver: Unknown hardware 
  Default Encoding    : windows-1252
  Deployed            : false
  Graphics card 1     : RealVNC ( 0x0 ) VNC Mirror Driver Version 1.8.0.0
  Graphics card 2     : NVIDIA ( 0x10de ) NVIDIA GeForce GTX 1080 Ti Version 23.21.13.8813
  Host Name           : ADRLAB-32
  MATLAB Architecture : win64
  MATLAB Entitlement ID: 3463632
  MATLAB Root         : C:\Program Files\MATLAB\R2017a
  MATLAB Version      : 9.2.0.556344 (R2017a)
  OpenGL              : hardware
  Operating System    : Microsoft Windows 7 Enterprise 
  Processor ID        : x86 Family 6 Model 60 Stepping 3, GenuineIntel
  Virtual Machine     : Java 1.7.0_60-b19 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode
  Window System       : Version 6.1 (Build 7601: Service Pack 1)

Fault Count: 1

Abnormal termination:
Unknown exception 0x40000015

Register State (from fault):
  RAX = 00000000730ce15e  RBX = 000007fee1c43b70
  RCX = 0000000004027df0  RDX = 00000000000000d8
  RSP = 0000000004028400  RBP = 00000000ffffffff
  RSI = 0000000004028600  RDI = 00000000338314e0

   R8 = 0000000000000000   R9 = 0000000000000000
  R10 = 43d6dde6d86822b7  R11 = 0000000004028440
  R12 = 0000000004028980  R13 = 0000000000000003
  R14 = 0000000000000000  R15 = 0000000000000000

  RIP = 000007fefcdca06d  EFL = 00000202

   CS = 0033   FS = 0053   GS = 002b

Stack Trace (from fault):
[  0] 0x000007fefcdca06d                 C:\Windows\system32\KERNELBASE.dll+00106605 RaiseException+00000061
[  1] 0x000007fee1c1d6bb    C:\ProgramData\Anaconda3\Library\bin\libzmq.dll+00186043
[  2] 0x000007fee1c03ffb    C:\ProgramData\Anaconda3\Library\bin\libzmq.dll+00081915
[  3] 0x000007fee1c21b88    C:\ProgramData\Anaconda3\Library\bin\libzmq.dll+00203656
[  4] 0x000007fee1c217c0    C:\ProgramData\Anaconda3\Library\bin\libzmq.dll+00202688
[  5] 0x000007fee1c35c48    C:\ProgramData\Anaconda3\Library\bin\libzmq.dll+00285768 zmq_msg_recv+00000040
[  6] 0x0000000092e114fb C:\Users\adkins\AppData\Local\Temp\tp0c88deb0_6b3e_493a_9c92_2fe78c8e2736\libzmq_thunk_pcwin64.dll+00005371 int32voidPtrvoidPtrint32Thunk+00000084
[  7] 0x0000000092d62792                             bin\win64\libmwcli.dll+00010130
[  8] 0x0000000092d6cdb4                             bin\win64\libmwcli.dll+00052660
[  9] 0x0000000092d710f5                             bin\win64\libmwcli.dll+00069877
[ 10] 0x0000000092d8c280                             bin\win64\libmwcli.dll+00180864 PointerMapSize+00084384
[ 11] 0x0000000092d8d34e                             bin\win64\libmwcli.dll+00185166 PointerMapSize+00088686
[ 12] 0x000000000c3ab0ee                     bin\win64\pgo\m_dispatcher.dll+00045294 mdLogging::~mdLogging+00000154
[ 13] 0x000000000c3a9be1                     bin\win64\pgo\m_dispatcher.dll+00039905 Mfh_MATLAB_fn::dispatch_fh+00000641
[ 14] 0x000000000d95a61c                            bin\win64\pgo\m_lxe.dll+00108060
[ 15] 0x000000000dab1ffb                            bin\win64\pgo\m_lxe.dll+01515515 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,std::vector<MathWorks::lxe::MatlabIrTree * __ptr64,std::allocator<MathWorks::lxe::MatlabIrTree * __ptr64> > >::load_object_data+00013323
[ 16] 0x000000000dab0aa7                            bin\win64\pgo\m_lxe.dll+01510055 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,std::vector<MathWorks::lxe::MatlabIrTree * __ptr64,std::allocator<MathWorks::lxe::MatlabIrTree * __ptr64> > >::load_object_data+00007863
[ 17] 0x000000000d95a717                            bin\win64\pgo\m_lxe.dll+00108311
[ 18] 0x000000000d9e9eb5                            bin\win64\pgo\m_lxe.dll+00695989
[ 19] 0x000000000d978e6a                            bin\win64\pgo\m_lxe.dll+00233066
[ 20] 0x000000000daba423                            bin\win64\pgo\m_lxe.dll+01549347 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,std::vector<MathWorks::lxe::MatlabIrTree * __ptr64,std::allocator<MathWorks::lxe::MatlabIrTree * __ptr64> > >::load_object_data+00047155
[ 21] 0x000000000db21a97                            bin\win64\pgo\m_lxe.dll+01972887 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,std::vector<MathWorks::lxe::MatlabIrTree * __ptr64,std::allocator<MathWorks::lxe::MatlabIrTree * __ptr64> > >::load_object_data+00470695
[ 22] 0x000000000db21a05                            bin\win64\pgo\m_lxe.dll+01972741 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,std::vector<MathWorks::lxe::MatlabIrTree * __ptr64,std::allocator<MathWorks::lxe::MatlabIrTree * __ptr64> > >::load_object_data+00470549
[ 23] 0x000000000d9626c9                            bin\win64\pgo\m_lxe.dll+00141001
[ 24] 0x000000000d9632f3                            bin\win64\pgo\m_lxe.dll+00144115
[ 25] 0x000000000d96474c                            bin\win64\pgo\m_lxe.dll+00149324
[ 26] 0x000000000d965288                            bin\win64\pgo\m_lxe.dll+00152200
[ 27] 0x000000000d96498f                            bin\win64\pgo\m_lxe.dll+00149903
[ 28] 0x000000000d958c4b                            bin\win64\pgo\m_lxe.dll+00101451
[ 29] 0x000000000d96004a                            bin\win64\pgo\m_lxe.dll+00131146
[ 30] 0x000000000d95f7d0                            bin\win64\pgo\m_lxe.dll+00128976
[ 31] 0x000000000dab5d0f                            bin\win64\pgo\m_lxe.dll+01531151 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,std::vector<MathWorks::lxe::MatlabIrTree * __ptr64,std::allocator<MathWorks::lxe::MatlabIrTree * __ptr64> > >::load_object_data+00028959
[ 32] 0x000000000dab52d3                            bin\win64\pgo\m_lxe.dll+01528531 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,std::vector<MathWorks::lxe::MatlabIrTree * __ptr64,std::allocator<MathWorks::lxe::MatlabIrTree * __ptr64> > >::load_object_data+00026339
[ 33] 0x000000000dab51b9                            bin\win64\pgo\m_lxe.dll+01528249 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,std::vector<MathWorks::lxe::MatlabIrTree * __ptr64,std::allocator<MathWorks::lxe::MatlabIrTree * __ptr64> > >::load_object_data+00026057
[ 34] 0x000000000db40ea9                            bin\win64\pgo\m_lxe.dll+02100905 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,MathWorks::lxe::MatlabIrTree>::load_object_data+00015609
[ 35] 0x000000000db40e32                            bin\win64\pgo\m_lxe.dll+02100786 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,MathWorks::lxe::MatlabIrTree>::load_object_data+00015490
[ 36] 0x000000000dab3383                            bin\win64\pgo\m_lxe.dll+01520515 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,std::vector<MathWorks::lxe::MatlabIrTree * __ptr64,std::allocator<MathWorks::lxe::MatlabIrTree * __ptr64> > >::load_object_data+00018323
[ 37] 0x000000000ca59aa9                    bin\win64\pgo\m_interpreter.dll+00170665 inEvalCmdWithLocalReturnInDesiredWSAndPublishEvents+00000081
[ 38] 0x00000000fcfe4926                                  bin\win64\iqm.dll+00346406 iqm::InternalEvalPlugin::inEvalCmdWithLocalReturn+00000246
[ 39] 0x00000000fcfe444d                                  bin\win64\iqm.dll+00345165 iqm::InternalEvalPlugin::execute+00000253
[ 40] 0x000000000d71d2e5                                  bin\win64\mcr.dll+00316133 mcrRegisterExternalFunction+00031957
[ 41] 0x00000000fcfd111a                                  bin\win64\iqm.dll+00266522 iqm::Iqm::setupIqmFcnPtrs+00072250
[ 42] 0x00000000fcfd0ff3                                  bin\win64\iqm.dll+00266227 iqm::Iqm::setupIqmFcnPtrs+00071955
[ 43] 0x00000000fcfb72da                                  bin\win64\iqm.dll+00160474 iqm::Iqm::deliver+00001114
[ 44] 0x00000000fb5fb7d5                          bin\win64\libmwbridge.dll+00047061 ioReadLine+00000517
[ 45] 0x00000000fb5fb692                          bin\win64\libmwbridge.dll+00046738 ioReadLine+00000194
[ 46] 0x00000000fb609dc9                          bin\win64\libmwbridge.dll+00105929 mnDebugPrompt+00001545
[ 47] 0x00000000fb609903                          bin\win64\libmwbridge.dll+00104707 mnDebugPrompt+00000323
[ 48] 0x00000000fb60a04a                          bin\win64\libmwbridge.dll+00106570 mnParser+00000426
[ 49] 0x000000000d7075f1                                  bin\win64\mcr.dll+00226801 mcr::runtime::setInterpreterThreadSingletonToCurrent+00027889
[ 50] 0x000000000d7064e7                                  bin\win64\mcr.dll+00222439 mcr::runtime::setInterpreterThreadSingletonToCurrent+00023527
[ 51] 0x000000000d706563                                  bin\win64\mcr.dll+00222563 mcr::runtime::setInterpreterThreadSingletonToCurrent+00023651
[ 52] 0x000000000d706e81                                  bin\win64\mcr.dll+00224897 mcr::runtime::setInterpreterThreadSingletonToCurrent+00025985
[ 53] 0x00000000fd01e647                                  bin\win64\iqm.dll+00583239 iqm::UserEvalPlugin::pre+00030695
[ 54] 0x00000000fd02af8c                                  bin\win64\iqm.dll+00634764 iqm::UserEvalPlugin::pre+00082220
[ 55] 0x00000000fd018770                                  bin\win64\iqm.dll+00558960 iqm::UserEvalPlugin::pre+00006416
[ 56] 0x00000000fd02de9a                                  bin\win64\iqm.dll+00646810 iqm::UserEvalPlugin::pre+00094266
[ 57] 0x00000000fcffad17                                  bin\win64\iqm.dll+00437527 iqm::PackagedTaskPlugin::PackagedTaskPlugin+00000727
[ 58] 0x00000000fcffb36f                                  bin\win64\iqm.dll+00439151 iqm::PackagedTaskPlugin::execute+00000575
[ 59] 0x00000000fcffad89                                  bin\win64\iqm.dll+00437641 iqm::PackagedTaskPlugin::PackagedTaskPlugin+00000841
[ 60] 0x00000000fcffb1e4                                  bin\win64\iqm.dll+00438756 iqm::PackagedTaskPlugin::execute+00000180
[ 61] 0x00000000fcfd111a                                  bin\win64\iqm.dll+00266522 iqm::Iqm::setupIqmFcnPtrs+00072250
[ 62] 0x00000000fcfd0ff3                                  bin\win64\iqm.dll+00266227 iqm::Iqm::setupIqmFcnPtrs+00071955
[ 63] 0x00000000fcfb7b95                                  bin\win64\iqm.dll+00162709 iqm::Iqm::deliver+00003349
[ 64] 0x00000000fcfb8815                                  bin\win64\iqm.dll+00165909 iqm::Iqm::deliver+00006549
[ 65] 0x0000000004786383                        bin\win64\libmwservices.dll+01074051 services::system_events::PpeDispatchHook::dispatchOne+00019811
[ 66] 0x000000000478abe3                        bin\win64\libmwservices.dll+01092579 sysq::addProcessPendingEventsUnitTestHook+00002099
[ 67] 0x000000000478add0                        bin\win64\libmwservices.dll+01093072 sysq::addProcessPendingEventsUnitTestHook+00002592
[ 68] 0x000000000478c095                        bin\win64\libmwservices.dll+01097877 sysq::getCondition+00003269
[ 69] 0x000000000478cf8f                        bin\win64\libmwservices.dll+01101711 svWS_ProcessPendingEvents+00000287
[ 70] 0x000000000d70795e                                  bin\win64\mcr.dll+00227678 mcr::runtime::setInterpreterThreadSingletonToCurrent+00028766
[ 71] 0x000000000d708046                                  bin\win64\mcr.dll+00229446 mcr::runtime::setInterpreterThreadSingletonToCurrent+00030534
[ 72] 0x000000000d6fe832                                  bin\win64\mcr.dll+00190514 mcr_process_events+00010210
[ 73] 0x000000000d700782                                  bin\win64\mcr.dll+00198530 mcr_process_events+00018226
[ 74] 0x000000000820c21e                             bin\win64\MVMLocal.dll+00246302 mvm_server::inproc::LocalFactory::terminate+00073982
[ 75] 0x00000000fa95a3d9                                  bin\win64\mvm.dll+01221593 mvm::detail::initLocalMvmHack+00000521
[ 76] 0x00000000fa95ab25                                  bin\win64\mvm.dll+01223461 mvm::detail::SessionImpl::privateSession+00000533
[ 77] 0x00000000fa95ad31                                  bin\win64\mvm.dll+01223985 mvm::detail::SessionImpl::privateSession+00001057
[ 78] 0x0000000140006fd5                               bin\win64\MATLAB.exe+00028629
[ 79] 0x0000000140007661                               bin\win64\MATLAB.exe+00030305
[ 80] 0x0000000076d159cd                   C:\Windows\system32\kernel32.dll+00088525 BaseThreadInitThunk+00000013
[ 81] 0x0000000076f7383d                      C:\Windows\SYSTEM32\ntdll.dll+00342077 RtlUserThreadStart+00000029

Versions

C:\Windows\system32>ipython --version
6.3.1
C:\Windows\system32>python --version
Python 3.5.5 :: Anaconda custom (64-bit)
C:\Windows\system32>conda --version
conda 4.5.1
C:\Windows\system32>pip show transplant
Name: Transplant
Version: 0.8.5
Summary: Call Matlab from Python (requires Matlab)
Home-page: https://github.com/bastibe/transplant
Author: Bastian Bechtold
Author-email: basti@bastibe.de
License: BSD 3-clause License
Location: c:\programdata\anaconda3\lib\site-packages
Requires: numpy, pyzmq, msgpack-python
C:\Windows\system32>conda list zeromq
# packages in environment at C:\ProgramData\Anaconda3:
#
# Name                    Version                   Build  Channel
zeromq                    4.2.3                hd6b2f15_3
>> mex -setup
MEX configured to use 'MinGW64 Compiler (C)' for C language compilation.
>> cc.Details

  CompilerConfigurationDetails with properties:

         CompilerExecutable: 'C:\ProgramData\MATLAB\SupportPackages\R2017a\3P.instrset\mingw_492.instrset\bin\g++'
              CompilerFlags: '-fexceptions -fno-omit-frame-pointer -std=c++11'
          OptimizationFlags: '-O -DNDEBUG'
                 DebugFlags: '-g'
           LinkerExecutable: 'C:\ProgramData\MATLAB\SupportPackages\R2017a\3P.instrset\mingw_492.instrset\bin\g++'
                LinkerFlags: '-m64 -Wl,--no-undefined -shared'
    LinkerOptimizationFlags: '-s'
           LinkerDebugFlags: '-g'
                     SetEnv: 'set COMPILER=C:\ProgramData\MATLAB\SupportPackages\R2017a\3P.instrset\mingw_492.instrset\bin\g++↵                set CCOMPILER=C:\ProgramData\MATLAB\SupportPackages\R2017a\3P.instrset\mingw_492.instrset\bin\gcc↵                set COMPFLAGS=-c -fexceptions -fno-omit-frame-pointer -std=c++11 -DTARGET_API_VERSION=700   -m64 -DMATLAB_MEX_FILE  -DMATLAB_MEX_FILE ↵                set CCOMPFLAGS=-c -fexceptions -fno-omit-frame-pointer -DTARGET_API_VERSION=700   -m64 -DMATLAB_MEX_FILE  -DMATLAB_MEX_FILE ↵                set OPTIMFLAGS=-O -DNDEBUG↵                set DEBUGFLAGS=-g↵                set LINKER=C:\ProgramData\MATLAB\SupportPackages\R2017a\3P.instrset\mingw_492.instrset\bin\g++↵                set CLINKER=C:\ProgramData\MATLAB\SupportPackages\R2017a\3P.instrset\mingw_492.instrset\bin\gcc↵                set LINKFLAGS=-m64 -Wl,--no-undefined -shared -L"C:\Program Files\MATLAB\R2017a\extern\lib\win64\mingw64" -llibmx -llibmex -llibmat -lm -llibmwlapack -llibmwblas -Wl,"C:\Program Files\MATLAB\R2017a/extern/lib/win64/mingw64/mexFunction.def"↵                set LINKDEBUGFLAGS=-g↵                set NAME_OUTPUT= -o "%OUTDIR%%MEX_NAME%%MEX_EXT%"↵set PATH=C:\ProgramData\MATLAB\SupportPackages\R2017a\3P.instrset\mingw_492.instrset\bin;C:\Program Files\MATLAB\R2017a\extern\include\win64;C:\Program Files\MATLAB\R2017a\extern\include;C:\Program Files\MATLAB\R2017a\simulink\include;C:\Program Files\MATLAB\R2017a\lib\win64;%MATLAB_BIN%;%PATH%↵set INCLUDE=C:\ProgramData\MATLAB\SupportPackages\R2017a\3P.instrset\mingw_492.instrset\include;%INCLUDE%↵set LIB=C:\ProgramData\MATLAB\SupportPackages\R2017a\3P.instrset\mingw_492.instrset\lib;;%LIB%↵set LIBPATH=C:\Program Files\MATLAB\R2017a\extern\lib\win64;%LIBPATH%↵'
           CommandLineShell: 'set MINGW_ROOT_PATH=C:\ProgramData\MATLAB\SupportPackages\R2017a\3P.instrset\mingw_492.instrset'
        CommandLineShellArg: ''
bastibe commented 6 years ago

Wow, thank you for this comprehensive bug report.

To summarize: Matlab seems to load the DLL correctly (evidenced by the crash dump stack trace line 4, where it calls into libzmq), but somehow crashes in zmq_msg_recv.

I currently don't have access to a Windows machine, so I can't reproduce this right now. My best hypothesis for now is that the msg object in ZMQ.m is somehow broken.

Could you try replacing the line

typedef struct zmq_msg_t {unsigned char hidden [64];} zmq_msg_t;

in transplantzmq.h with the actual quote from zmq.h

typedef struct zmq_msg_t {
#if defined (__GNUC__) || defined ( __INTEL_COMPILER) || \
        (defined (__SUNPRO_C) && __SUNPRO_C >= 0x590) || \
        (defined (__SUNPRO_CC) && __SUNPRO_CC >= 0x590)
    unsigned char hidden [64] __attribute__ ((aligned (sizeof (void *))));
#elif defined (_MSC_VER) && (defined (_M_X64) || defined (_M_ARM64))
    __declspec (align (8)) unsigned char hidden [64];
#elif defined (_MSC_VER) && (defined (_M_IX86) || defined (_M_ARM_ARMV7VE))
    __declspec (align (4)) unsigned char hidden [64];
#else
    unsigned char hidden [64];
#endif

and see if that fixes the issue? (I renamed _ to hidden because Matlab can't create a struct with a field named _). Maybe that pointer-alignment is causing problems.

zanieb commented 6 years ago

Copying the section you've provided gave the same error (note you actually excluded the final portion of the typedef, so I added it)

typedef struct zmq_msg_t {
#if defined (__GNUC__) || defined ( __INTEL_COMPILER) || \
        (defined (__SUNPRO_C) && __SUNPRO_C >= 0x590) || \
        (defined (__SUNPRO_CC) && __SUNPRO_CC >= 0x590)
    unsigned char hidden [64] __attribute__ ((aligned (sizeof (void *))));
#elif defined (_MSC_VER) && (defined (_M_X64) || defined (_M_ARM64))
    __declspec (align (8)) unsigned char hidden [64];
#elif defined (_MSC_VER) && (defined (_M_IX86) || defined (_M_ARM_ARMV7VE))
    __declspec (align (4)) unsigned char hidden [64];
#else
    unsigned char hidden [64];
#endif
} zmq_msg_t;

If I copy the entire zmq.h -> transplantzmq.h and replace _ with hidden, I get the following error

To get started, type one of these: helpwin, helpdesk, or demo.
For product information, visit www.mathworks.com.

Warning: The data type ' unsigned char hidden [ 64 ] __attribute__ (( aligned ( sizeof ( void *))))' used by structure zmq_msg_t does
not exist.  The structure may not be usable. 
> In loadlibrary
  In ZMQ (line 25)
  In transplant_remote (line 43) 
Warning: The data type 'FcnPtrPtr' used by function zmq_msg_init_data does not exist. 
> In loadlibrary
  In ZMQ (line 25)
  In transplant_remote (line 43) 
Warning: The function 'zmq_msg_gets' was not found in the library 
> In loadlibrary
  In ZMQ (line 25)
  In transplant_remote (line 43) 
Warning: The function 'zmq_proxy_steerable' was not found in the library 
> In loadlibrary
  In ZMQ (line 25)
  In transplant_remote (line 43) 
Warning: The function 'zmq_has' was not found in the library 
> In loadlibrary
  In ZMQ (line 25)
  In transplant_remote (line 43) 
Warning: The data type 'FcnPtrPtr' used by function zmq_threadstart does not exist. 
> In loadlibrary
  In ZMQ (line 25)
  In transplant_remote (line 43) 
Error loading libzmq: Could not load ZMQ library
Warning: The following error was caught while executing 'ZMQ' class destructor:
Error using ZMQ/delete (line 73)
Error in zmq_close: Unknown error 128

Error in ZMQ (line 20)
        function obj = ZMQ(libname, address)

Error in transplant_remote (line 43)
            messenger = ZMQ(zmqname, url); 
> In ZMQ (line 20)
  In transplant_remote (line 43) 
»

The unknown error (128) appears to be from it closing while no context exists.

When explored in the Matlab debugger, it became clear that the not_found variable was being set (library load error). I figured there may be a mismatch in versions so I tried the Anaconda 0MQ again and explored with the Matlab debugger in the ZMQ constructor. This loads the library correctly, but the zmq_msg_t warning seems critical.

>>mq = ZMQ('C:\\ProgramData\\Anaconda3\\Library\\bin\\libzmq.dll', 'tcp://127.0.0.1:65535')
Warning: The data type ' unsigned char hidden [ 64 ] __attribute__ (( aligned ( sizeof ( void *))))' used by structure zmq_msg_t does not exist.  The
structure may not be usable. 
> In loadlibrary
  In ZMQ (line 25) 
Warning: The data type 'FcnPtrPtr' used by function zmq_msg_init_data does not exist. 
> In loadlibrary
  In ZMQ (line 25) 
Warning: The data type 'FcnPtrPtr' used by function zmq_threadstart does not exist. 
> In loadlibrary
  In ZMQ (line 25) 
>> mq.send([1,2,3,4])
Error using ZMQ/send (line 64)
Error in zmq_send: The *zmq_send()* operation cannot be performed on this socket at the moment due to the socket not being in the appropriate state.

I thought a lack of data type definition for unsigned char may be the problem, but typedef unsigned char u08; still results in process death. Perhaps aligned is not defined? I'm not experienced with C, but the syntax appears to be correct for gcc. Additionally, it appears the socket error may be important although it passes the not null assertion.

I'm attempting to switch from MinGW to VC15 to see if that resolves anything since I haven't used MSC with the alignment code.

bastibe commented 6 years ago

From a cursory internet search, it appears that the error message "Warning: The data type ... used by structure zmq_msg_t does not exist. The structure may not be usable." is generated by loadlibrary (as opposed to GCC), and seems to indicate that loadlibrary can't parse some part of the definition.

I'm assuming that the problem is the alignment. Which means we will have to cheat to enforce alignment:

Replace the zmq_msg_t definition in transplantzmq.h:16 with

typedef struct zmq_msg_t {size_t hidden [8];} zmq_msg_t;

And the instantiation in ZMQ.h:46 with

msg = libstruct('zmq_msg_t', struct('hidden', zeros(1, 8, 'uint64')));

This will define zmq_msg_t as a structure containing the same number of bytes as before, but since its contents are now 64-bit values, the compiler will align them to 64 bits.

This hack does not break transplant on my computer. Let's see if it fixes the issue on yours!

(Just as a sanity check: Both Matlab and libzmq are 64 bit on your computer, right?)

zanieb commented 6 years ago

They're both for sure 64-bit.

With those changes (the second in ZMQ.m:46), the process still dies. The warning is gone and we're back to just a stack trace from Matlab's crash (which appears to be the same as before)

------------------------------------------------------------------------
    Unknown exception 0x40000015 detected at Mon Apr 23 12:34:19 2018
------------------------------------------------------------------------

Configuration:
  Crash Decoding      : Disabled - No sandbox or build area path
  Crash Mode          : continue (default)
  Current Graphics Driver: Unknown hardware 
  Default Encoding    : windows-1252
  Deployed            : false
  Graphics card 1     : RealVNC ( 0x0 ) VNC Mirror Driver Version 1.8.0.0
  Graphics card 2     : NVIDIA ( 0x10de ) NVIDIA GeForce GTX 1080 Ti Version 23.21.13.8813
  Host Name           : ADRLAB-32
  MATLAB Architecture : win64
  MATLAB Entitlement ID: 3463632
  MATLAB Root         : C:\Program Files\MATLAB\R2017a
  MATLAB Version      : 9.2.0.556344 (R2017a)
  OpenGL              : hardware
  Operating System    : Microsoft Windows 7 Enterprise 
  Processor ID        : x86 Family 6 Model 60 Stepping 3, GenuineIntel
  Virtual Machine     : Java 1.7.0_60-b19 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode
  Window System       : Version 6.1 (Build 7601: Service Pack 1)

Fault Count: 1

Abnormal termination:
Unknown exception 0x40000015

Register State (from fault):
  RAX = 000000006761eaee  RBX = 000007fee27e3b70
  RCX = 0000000004027df0  RDX = 00000000000000d8
  RSP = 0000000004028400  RBP = 00000000ffffffff
  RSI = 0000000004028600  RDI = 00000000339c39e0

   R8 = 0000000000000000   R9 = 0000000000000000
  R10 = 051e61ca59b5349f  R11 = 0000000004028440
  R12 = 0000000004028980  R13 = 0000000000000003
  R14 = 0000000000000000  R15 = 0000000000000000

  RIP = 000007fefd15a06d  EFL = 00000206

   CS = 0033   FS = 0053   GS = 002b

Stack Trace (from fault):
[  0] 0x000007fefd15a06d                 C:\Windows\system32\KERNELBASE.dll+00106605 RaiseException+00000061
[  1] 0x000007fee27bd6bb    C:\ProgramData\Anaconda3\Library\bin\libzmq.dll+00186043
[  2] 0x000007fee27a3ffb    C:\ProgramData\Anaconda3\Library\bin\libzmq.dll+00081915
[  3] 0x000007fee27c1b88    C:\ProgramData\Anaconda3\Library\bin\libzmq.dll+00203656
[  4] 0x000007fee27c17c0    C:\ProgramData\Anaconda3\Library\bin\libzmq.dll+00202688
[  5] 0x000007fee27d5c48    C:\ProgramData\Anaconda3\Library\bin\libzmq.dll+00285768 zmq_msg_recv+00000040
[  6] 0x000000006faf14fb C:\Users\adkins\AppData\Local\Temp\tp82b477d1_6518_4e97_a196_75180b2e2adc\libzmq_thunk_pcwin64.dll+00005371 int32voidPtrvoidPtrint32Thunk+00000084
[  7] 0x00000000e4372792                             bin\win64\libmwcli.dll+00010130
[  8] 0x00000000e437cdb4                             bin\win64\libmwcli.dll+00052660
[  9] 0x00000000e43810f5                             bin\win64\libmwcli.dll+00069877
[ 10] 0x00000000e439c280                             bin\win64\libmwcli.dll+00180864 PointerMapSize+00084384
[ 11] 0x00000000e439d34e                             bin\win64\libmwcli.dll+00185166 PointerMapSize+00088686
[ 12] 0x000000000c6eb0ee                     bin\win64\pgo\m_dispatcher.dll+00045294 mdLogging::~mdLogging+00000154
[ 13] 0x000000000c6e9be1                     bin\win64\pgo\m_dispatcher.dll+00039905 Mfh_MATLAB_fn::dispatch_fh+00000641
[ 14] 0x000000000dc6a61c                            bin\win64\pgo\m_lxe.dll+00108060
[ 15] 0x000000000ddc1ffb                            bin\win64\pgo\m_lxe.dll+01515515 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,std::vector<MathWorks::lxe::MatlabIrTree * __ptr64,std::allocator<MathWorks::lxe::MatlabIrTree * __ptr64> > >::load_object_data+00013323
[ 16] 0x000000000ddc0aa7                            bin\win64\pgo\m_lxe.dll+01510055 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,std::vector<MathWorks::lxe::MatlabIrTree * __ptr64,std::allocator<MathWorks::lxe::MatlabIrTree * __ptr64> > >::load_object_data+00007863
[ 17] 0x000000000dc6a717                            bin\win64\pgo\m_lxe.dll+00108311
[ 18] 0x000000000dcf9eb5                            bin\win64\pgo\m_lxe.dll+00695989
[ 19] 0x000000000dc88e6a                            bin\win64\pgo\m_lxe.dll+00233066
[ 20] 0x000000000ddca423                            bin\win64\pgo\m_lxe.dll+01549347 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,std::vector<MathWorks::lxe::MatlabIrTree * __ptr64,std::allocator<MathWorks::lxe::MatlabIrTree * __ptr64> > >::load_object_data+00047155
[ 21] 0x000000000de31a97                            bin\win64\pgo\m_lxe.dll+01972887 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,std::vector<MathWorks::lxe::MatlabIrTree * __ptr64,std::allocator<MathWorks::lxe::MatlabIrTree * __ptr64> > >::load_object_data+00470695
[ 22] 0x000000000de31a05                            bin\win64\pgo\m_lxe.dll+01972741 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,std::vector<MathWorks::lxe::MatlabIrTree * __ptr64,std::allocator<MathWorks::lxe::MatlabIrTree * __ptr64> > >::load_object_data+00470549
[ 23] 0x000000000dc726c9                            bin\win64\pgo\m_lxe.dll+00141001
[ 24] 0x000000000dc732f3                            bin\win64\pgo\m_lxe.dll+00144115
[ 25] 0x000000000dc7474c                            bin\win64\pgo\m_lxe.dll+00149324
[ 26] 0x000000000dc75288                            bin\win64\pgo\m_lxe.dll+00152200
[ 27] 0x000000000dc7498f                            bin\win64\pgo\m_lxe.dll+00149903
[ 28] 0x000000000dc68c4b                            bin\win64\pgo\m_lxe.dll+00101451
[ 29] 0x000000000dc7004a                            bin\win64\pgo\m_lxe.dll+00131146
[ 30] 0x000000000dc6f7d0                            bin\win64\pgo\m_lxe.dll+00128976
[ 31] 0x000000000ddc5d0f                            bin\win64\pgo\m_lxe.dll+01531151 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,std::vector<MathWorks::lxe::MatlabIrTree * __ptr64,std::allocator<MathWorks::lxe::MatlabIrTree * __ptr64> > >::load_object_data+00028959
[ 32] 0x000000000ddc52d3                            bin\win64\pgo\m_lxe.dll+01528531 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,std::vector<MathWorks::lxe::MatlabIrTree * __ptr64,std::allocator<MathWorks::lxe::MatlabIrTree * __ptr64> > >::load_object_data+00026339
[ 33] 0x000000000ddc51b9                            bin\win64\pgo\m_lxe.dll+01528249 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,std::vector<MathWorks::lxe::MatlabIrTree * __ptr64,std::allocator<MathWorks::lxe::MatlabIrTree * __ptr64> > >::load_object_data+00026057
[ 34] 0x000000000de50ea9                            bin\win64\pgo\m_lxe.dll+02100905 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,MathWorks::lxe::MatlabIrTree>::load_object_data+00015609
[ 35] 0x000000000de50e32                            bin\win64\pgo\m_lxe.dll+02100786 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,MathWorks::lxe::MatlabIrTree>::load_object_data+00015490
[ 36] 0x000000000ddc3383                            bin\win64\pgo\m_lxe.dll+01520515 boost::archive::detail::iserializer<boost::archive::binaryTerm_iarchive,std::vector<MathWorks::lxe::MatlabIrTree * __ptr64,std::allocator<MathWorks::lxe::MatlabIrTree * __ptr64> > >::load_object_data+00018323
[ 37] 0x000000000cd69aa9                    bin\win64\pgo\m_interpreter.dll+00170665 inEvalCmdWithLocalReturnInDesiredWSAndPublishEvents+00000081
[ 38] 0x00000000fcfe4926                                  bin\win64\iqm.dll+00346406 iqm::InternalEvalPlugin::inEvalCmdWithLocalReturn+00000246
[ 39] 0x00000000fcfe444d                                  bin\win64\iqm.dll+00345165 iqm::InternalEvalPlugin::execute+00000253
[ 40] 0x000000000da2d2e5                                  bin\win64\mcr.dll+00316133 mcrRegisterExternalFunction+00031957
[ 41] 0x00000000fcfd111a                                  bin\win64\iqm.dll+00266522 iqm::Iqm::setupIqmFcnPtrs+00072250
[ 42] 0x00000000fcfd0ff3                                  bin\win64\iqm.dll+00266227 iqm::Iqm::setupIqmFcnPtrs+00071955
[ 43] 0x00000000fcfb72da                                  bin\win64\iqm.dll+00160474 iqm::Iqm::deliver+00001114
[ 44] 0x00000000fb5fb7d5                          bin\win64\libmwbridge.dll+00047061 ioReadLine+00000517
[ 45] 0x00000000fb5fb692                          bin\win64\libmwbridge.dll+00046738 ioReadLine+00000194
[ 46] 0x00000000fb609dc9                          bin\win64\libmwbridge.dll+00105929 mnDebugPrompt+00001545
[ 47] 0x00000000fb609903                          bin\win64\libmwbridge.dll+00104707 mnDebugPrompt+00000323
[ 48] 0x00000000fb60a04a                          bin\win64\libmwbridge.dll+00106570 mnParser+00000426
[ 49] 0x000000000da175f1                                  bin\win64\mcr.dll+00226801 mcr::runtime::setInterpreterThreadSingletonToCurrent+00027889
[ 50] 0x000000000da164e7                                  bin\win64\mcr.dll+00222439 mcr::runtime::setInterpreterThreadSingletonToCurrent+00023527
[ 51] 0x000000000da16563                                  bin\win64\mcr.dll+00222563 mcr::runtime::setInterpreterThreadSingletonToCurrent+00023651
[ 52] 0x000000000da16e81                                  bin\win64\mcr.dll+00224897 mcr::runtime::setInterpreterThreadSingletonToCurrent+00025985
[ 53] 0x00000000fd01e647                                  bin\win64\iqm.dll+00583239 iqm::UserEvalPlugin::pre+00030695
[ 54] 0x00000000fd02af8c                                  bin\win64\iqm.dll+00634764 iqm::UserEvalPlugin::pre+00082220
[ 55] 0x00000000fd018770                                  bin\win64\iqm.dll+00558960 iqm::UserEvalPlugin::pre+00006416
[ 56] 0x00000000fd02de9a                                  bin\win64\iqm.dll+00646810 iqm::UserEvalPlugin::pre+00094266
[ 57] 0x00000000fcffad17                                  bin\win64\iqm.dll+00437527 iqm::PackagedTaskPlugin::PackagedTaskPlugin+00000727
[ 58] 0x00000000fcffb36f                                  bin\win64\iqm.dll+00439151 iqm::PackagedTaskPlugin::execute+00000575
[ 59] 0x00000000fcffad89                                  bin\win64\iqm.dll+00437641 iqm::PackagedTaskPlugin::PackagedTaskPlugin+00000841
[ 60] 0x00000000fcffb1e4                                  bin\win64\iqm.dll+00438756 iqm::PackagedTaskPlugin::execute+00000180
[ 61] 0x00000000fcfd111a                                  bin\win64\iqm.dll+00266522 iqm::Iqm::setupIqmFcnPtrs+00072250
[ 62] 0x00000000fcfd0ff3                                  bin\win64\iqm.dll+00266227 iqm::Iqm::setupIqmFcnPtrs+00071955
[ 63] 0x00000000fcfb7b95                                  bin\win64\iqm.dll+00162709 iqm::Iqm::deliver+00003349
[ 64] 0x00000000fcfb8815                                  bin\win64\iqm.dll+00165909 iqm::Iqm::deliver+00006549
[ 65] 0x0000000004776383                        bin\win64\libmwservices.dll+01074051 services::system_events::PpeDispatchHook::dispatchOne+00019811
[ 66] 0x000000000477abe3                        bin\win64\libmwservices.dll+01092579 sysq::addProcessPendingEventsUnitTestHook+00002099
[ 67] 0x000000000477add0                        bin\win64\libmwservices.dll+01093072 sysq::addProcessPendingEventsUnitTestHook+00002592
[ 68] 0x000000000477c095                        bin\win64\libmwservices.dll+01097877 sysq::getCondition+00003269
[ 69] 0x000000000477cf8f                        bin\win64\libmwservices.dll+01101711 svWS_ProcessPendingEvents+00000287
[ 70] 0x000000000da1795e                                  bin\win64\mcr.dll+00227678 mcr::runtime::setInterpreterThreadSingletonToCurrent+00028766
[ 71] 0x000000000da18046                                  bin\win64\mcr.dll+00229446 mcr::runtime::setInterpreterThreadSingletonToCurrent+00030534
[ 72] 0x000000000da0e832                                  bin\win64\mcr.dll+00190514 mcr_process_events+00010210
[ 73] 0x000000000da10782                                  bin\win64\mcr.dll+00198530 mcr_process_events+00018226
[ 74] 0x000000000c65c21e                             bin\win64\MVMLocal.dll+00246302 mvm_server::inproc::LocalFactory::terminate+00073982
[ 75] 0x00000000fa95a3d9                                  bin\win64\mvm.dll+01221593 mvm::detail::initLocalMvmHack+00000521
[ 76] 0x00000000fa95ab25                                  bin\win64\mvm.dll+01223461 mvm::detail::SessionImpl::privateSession+00000533
[ 77] 0x00000000fa95ad31                                  bin\win64\mvm.dll+01223985 mvm::detail::SessionImpl::privateSession+00001057
[ 78] 0x0000000140006fd5                               bin\win64\MATLAB.exe+00028629
[ 79] 0x0000000140007661                               bin\win64\MATLAB.exe+00030305
[ 80] 0x00000000770459cd                   C:\Windows\system32\kernel32.dll+00088525 BaseThreadInitThunk+00000013
[ 81] 0x00000000771a383d                      C:\Windows\SYSTEM32\ntdll.dll+00342077 RtlUserThreadStart+00000029
bastibe commented 6 years ago

I searched the internet for the error code 0x40000015, and it seems to be associated with some system files being corrupted.

That said, I tried to reproduce your error and set up a Windows virtual machine, and it crashes there, as well (though without an error message). I'll investigate further.

bastibe commented 6 years ago

The error on my machine can be fixed by starting Matlab with transplant.Matlab(arguments=['-wait']). Does this fix your problem as well?

bastibe commented 6 years ago

I just updated Transplant with this option enabled by default.

zanieb commented 6 years ago

Sorry about the delay, I had moved back to exclusively Linux to get my work done. This is now tested and working on my Windows machine. Thank you so much.

bastibe commented 6 years ago

Thank you for bringing it up! This is actually a regression I introduced a while ago and didn't notice (because the Matlab documentation didn't mention it anywhere).