search

andyschwarzl / gpuNUFFT

gpuNUFFT - An Open-Source GPU Library for 3D Gridding with Direct Matlab and Python Interface
http://cai2r.net/resources/software/gpunufft-open-source-gpu-library-3d-gridding-direct-matlab-interface
Other
77 stars 32 forks source link

Out-of-memory in gpuNUFFT/CUDA/inc/cuda_utils.hpp at line 40 #68

Open fyrdahl opened 6 years ago

fyrdahl commented 6 years ago

I'm experiencing sporadic crashes with an out-of-memory error message that points towards cuda_utils.hpp line 40. Running on Nvidia Tesla K40M (12Gb GDDR5) and I'm working on 2D data, so I'm nowhere near capacity. Could this be a memory leak?

out of memory in /home/alex/gpuNUFFT/CUDA/inc/cuda_utils.hpp at line 40

andyschwarzl commented 6 years ago

I'm sorry that you are experiencing crashes.

Can you give me some more details about your setup and data structure (OS, data/grid dimensions)?

Are you using the Matlab wrapper or a standalone C++ application?

fyrdahl commented 6 years ago

Of course! I'm using the Matlab wrapper to grid about 36 radial spokes at the time to a 128 matrix. If by OS you mean oversampling it's 1.5 and if you mean operating system it's Ubuntu 14.04. I used a kernel size 3 and when I looked into it I noticed I was using an unusually small sector width of 3.

I ran a batch job with 40216 images reconstructed with 16 iterations of CG-SENSE, so that would be 1286912 gridding operations (forward and adjoint). In that run I experienced the "out of memory" error about 5 times, so that should give an idea of how often it occurs.

andyschwarzl commented 6 years ago

Thanks for sharing the information. Sector width of 3 is indeed small, but should basically work.

The batch job is quite large - are you reusing the Gridding operator instance inside Matlab?

I've also experienced issues with huge batch jobs where GPU memory fragmentation caused these "out of memory" errors, although enough device memory had been available.

fyrdahl commented 6 years ago

No, 40216 unique gridding operators. I do clear them in between though. I was thinking it might be a memory leak, but you're saying about memory fragmentation makes sense.

Granted this is not a normal use case, we might be able to write it off as "shit happens" and close the issue?

andyschwarzl commented 6 years ago

Ok, no, I wouldn't just close it ;)

Is every trajectory unique in your 40k measurements?

So what I would try is to reuse existing operators or to allocate multiple operators at once...

fyrdahl commented 6 years ago

They are all unique. Binned reconstructions, the sheer amount is due to the fact that I'm batching a year worth of acquisitions.

New error that is more descriptive;

cufft has failed at adj with err 6 memory usage, free: 9687040 total: 12079136768 .out of memory in /home/alex/gpuNUFFT/CUDA/inc/cuda_utils.hpp at line 40

schrau24 commented 6 years ago

Was there ever a resolution to this? I have encountered a potentially similar out of memory issue recently when reconstructing a single 2D slice from 3D data (imageDim goes from [128 128 128] to [128 1 128]). This seems to work in most cases, however, when I repeatedly call the gpuNUFFT object it crashes and closes Matlab. I have discovered this happens when mex_gpuNUFFT_adj_atomic_f is called, but unfortunately because it quickly crashes Matlab I do not get any feedback on the what the problem is. I have tested it using the same data with different sizes and it seems to happen arbitrarily.

andyschwarzl commented 6 years ago

No, I didn't have the time to tackle the problem. Can you provide me with a simple reconstruction script that (arbitrarily) reproduces the behavior?

Mrvolcan commented 5 years ago

I am having the same problem when reconstructing several datasets after each other. GPU memory is not fully cleared after each reconstruction. It adds up and after several datasets reaches the maximum of the GPU memory, which then leads to a crash. Only solution so far is closing matlab after each dataset.

fyrdahl commented 5 years ago

I falsely believed that running clear gpuNUFFT would be sufficient. I later found out that I in order to release all memory, I had to clear the mex objects explicitly. The following seemed to solve the problem for me: clear @gpuNUFFT/private/mex_gpuNUFFT_precomp_f; clear @gpuNUFFT/private/mex_gpuNUFFT_adj_atomic_f;

Mrvolcan commented 5 years ago

Thanks a lot. That solved my problem.

andyschwarzl commented 5 years ago

Thanks for the follow-up. To be consistent, one should also clear

clear @gpuNUFFT/private/mex_gpuNUFFT_forw_atomic_f

curtcorum commented 4 years ago

Working through a similar issue:

curt@green:~/ngfn_simulation_recon/matlab$ nice matlab Gtk-Message: 17:45:22.504: Failed to load module "canberra-gtk-module" out of memory in /home/curt/Dropbox/ngfn_simulation_recon/matlab/gpuNUFFT/CUDA/inc/cuda_utils.hpp at line 40 corrupted double-linked list Segmentation fault (core dumped)

At this point trying to figure out if the options settings have any effect (note OS = 1.25, KW = 2.5, imageDim = [480 490 480]):

%FT_i = gpuNUFFT( kspace_i', dcf_d.^2, OS, KW, 8, imageDim, [], atomic, textures, balanced);
FT_i = gpuNUFFT( kspace_i', dcf_d.^2, OS, KW, 8, imageDim, []);

It should not be an actual out of memory, although it is a big 3d dataset and gridded k-space (estimate total approaching 1GB). It seems that the memory usage is a fair amount larger than just the data and oversampled gridding space (multiple copies made in gpu?).

any ideas appreciated, Curt

curt@green:~$ nvidia-smi Fri Aug 7 17:58:55 2020
+-----------------------------------------------------------------------------+ | NVIDIA-SMI 440.100 Driver Version: 440.100 CUDA Version: 10.2 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===============================+======================+======================| | 0 Quadro RTX 8000 Off | 00000000:01:00.0 On | Off | | 34% 61C P0 79W / 260W | 586MiB / 48596MiB | 3% Default | +-------------------------------+----------------------+----------------------+ | 1 Quadro RTX 8000 Off | 00000000:4B:00.0 Off | Off | | 33% 38C P8 29W / 260W | 323MiB / 48601MiB | 0% Default | +-------------------------------+----------------------+----------------------+

curt@green:~$ cat matlab_crash_dump.97123-1 

           abort() detected at Fri Aug 07 17:47:28 2020 -0500

Configuration: Crash Decoding : Disabled - No sandbox or build area path Crash Mode : continue (default) Default Encoding : UTF-8 Deployed : false Desktop Environment : GNOME-Flashback:GNOME GNU C Library : 2.27 stable Graphics Driver : Unknown hardware Graphics card 1 : 0x10de ( 0x10de ) 0x1e30 Version 440.100.0.0 (0-0-0) Graphics card 2 : 0x10de ( 0x10de ) 0x1e30 Version 440.100.0.0 (0-0-0) Java Version : Java 1.8.0_202-b08 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode MATLAB Architecture : glnxa64 MATLAB Entitlement ID : 4651242 MATLAB Root : /usr/local/MATLAB/R2019b MATLAB Version : 9.7.0.1319299 (R2019b) Update 5 OpenGL : hardware Operating System : Ubuntu 18.04.4 LTS Process ID : 97123 Processor ID : x86 Family 143 Model 49 Stepping 0, AuthenticAMD Session Key : a02974d2-4df2-41e8-8cc3-8bfaa4c455bc Static TLS mitigation : Enabled: Full Window System : The X.Org Foundation (12005000), display :1

Fault Count: 1

Abnormal termination: abort()

Register State (from fault): RAX = 0000000000000000 RBX = 00007fe0ea0add80 RCX = 00007fe1062a4f47 RDX = 0000000000000000 RSP = 00007fe0ea0adb10 RBP = 00007fe0ea0ade80 RSI = 00007fe0ea0adb10 RDI = 0000000000000002

R8 = 0000000000000000 R9 = 00007fe0ea0adb10 R10 = 0000000000000008 R11 = 0000000000000246 R12 = 00007fe0ea0add80 R13 = 0000000000001000 R14 = 0000000000000000 R15 = 0000000000000030

RIP = 00007fe1062a4f47 EFL = 0000000000000246

CS = 0033 FS = 0000 GS = 0000

Stack Trace (from fault): [ 0] 0x00007fe1062a4f47 /lib/x86_64-linux-gnu/libc.so.6+00257863 gsignal+00000199 [ 1] 0x00007fe1062a68b1 /lib/x86_64-linux-gnu/libc.so.6+00264369 abort+00000321 [ 2] 0x00007fe1062ef907 /lib/x86_64-linux-gnu/libc.so.6+00563463 [ 3] 0x00007fe1062f697a /lib/x86_64-linux-gnu/libc.so.6+00592250 [ 4] 0x00007fe1062f6b34 /lib/x86_64-linux-gnu/libc.so.6+00592692 [ 5] 0x00007fe1062fe0ab /lib/x86_64-linux-gnu/libc.so.6+00622763 cfree+00001771 [ 6] 0x00007fe1062a90f1 /lib/x86_64-linux-gnu/libc.so.6+00274673 [ 7] 0x00007fe1062a91ea /lib/x86_64-linux-gnu/libc.so.6+00274922 [ 8] 0x00007fdc89519ca6 /home/curt/Dropbox/ngfn_simulation_recon/matlab/gpuNUFFT/CUDA/bin/libgpuNUFFT_f.so+00179366 _Z17allocateDeviceMemI6float2EvPPT_j+00000125 [ 9] 0x00007fdc8951e554 /home/curt/Dropbox/ngfn_simulation_recon/matlab/gpuNUFFT/CUDA/bin/libgpuNUFFT_f.so+00197972 _Z25performTextureConvolutionP6float2PfS0_S1_PjS2_PN8gpuNUFFT12GpuNUFFTInfoE+00000105 [ 10] 0x00007fdc8951a45e /home/curt/Dropbox/ngfn_simulation_recon/matlab/gpuNUFFT/CUDA/bin/libgpuNUFFT_f.so+00181342 _ZN8gpuNUFFT23TextureGpuNUFFTOperator14adjConvolutionEP6float2PfS2_S3_PjS4_PNS_12GpuNUFFTInfoE+00000124 [ 11] 0x00007fdc89517cba /home/curt/Dropbox/ngfn_simulation_recon/matlab/gpuNUFFT/CUDA/bin/libgpuNUFFT_f.so+00171194 _ZN8gpuNUFFT16GpuNUFFTOperator18performGpuNUFFTAdjENS_5ArrayI6float2EERS3_NS_14GpuNUFFTOutputE+00000930 [ 12] 0x00007fdc89518340 /home/curt/Dropbox/ngfn_simulation_recon/matlab/gpuNUFFT/CUDA/bin/libgpuNUFFT_f.so+00172864 _ZN8gpuNUFFT16GpuNUFFTOperator18performGpuNUFFTAdjENS_5ArrayI6float2EENS_14GpuNUFFTOutputE+00000308 [ 13] 0x00007fdc8950dbf2 /home/curt/Dropbox/ngfn_simulation_recon/matlab/gpuNUFFT/CUDA/bin/libgpuNUFFT_f.so+00130034 _ZN8gpuNUFFT23GpuNUFFTOperatorFactory28computeDeapodizationFunctionERKjRKfRNS_10DimensionsE+00001200 [ 14] 0x00007fdc8950e7f8 /home/curt/Dropbox/ngfn_simulation_recon/matlab/gpuNUFFT/CUDA/bin/libgpuNUFFT_f.so+00133112 _ZN8gpuNUFFT23GpuNUFFTOperatorFactory22createGpuNUFFTOperatorERNS_5ArrayIfEES3_RNS1_I6float2EERKjS8_RKfRNS_10DimensionsE+00002162 [ 15] 0x00007fdc8b1781b7 /home/curt/Dropbox/ngfn_simulation_recon/matlab/gpuNUFFT/gpuNUFFT/@gpuNUFFT/private/mex_gpuNUFFT_precomp_f.mexa64+00041399 _ZN8gpuNUFFT29GpuNUFFTOperatorMatlabFactory22createGpuNUFFTOperatorERNS_5ArrayIfEES3_RNS1_I6float2EERKjS8_RKfRNS_10DimensionsEPP11mxArray_tag+00000089 [ 16] 0x00007fdc8b177173 /home/curt/Dropbox/ngfn_simulation_recon/matlab/gpuNUFFT/gpuNUFFT/@gpuNUFFT/private/mex_gpuNUFFT_precomp_f.mexa64+00037235 mexFunction+00000698 [ 17] 0x00007fe0f1ecede3 /usr/local/MATLAB/R2019b/bin/glnxa64/libmex.so+00548323 [ 18] 0x00007fe0f1eceee5 /usr/local/MATLAB/R2019b/bin/glnxa64/libmex.so+00548581 [ 19] 0x00007fe0f1ecf317 /usr/local/MATLAB/R2019b/bin/glnxa64/libmex.so+00549655 [ 20] 0x00007fe0f1ecff93 /usr/local/MATLAB/R2019b/bin/glnxa64/libmex.so+00552851 [ 21] 0x00007fe0f1ebb7ac /usr/local/MATLAB/R2019b/bin/glnxa64/libmex.so+00468908 [ 22] 0x00007fe0f2d2115f /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_dispatcher.so+01073503 _ZN8Mfh_file20dispatch_file_commonEMS_FviPP11mxArray_tagiS2_EiS2iS2+00000207 [ 23] 0x00007fe0f2d22c5e /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_dispatcher.so+01080414 [ 24] 0x00007fe0f2d231a1 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_dispatcher.so+01081761 _ZN8Mfh_file8dispatchEiPSt10unique_ptrI11mxArray_tagN6matrix6detail17mxDestroydeleterEEiPPS1+00000033 [ 25] 0x00007fe0f0344a63 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+14006883 [ 26] 0x00007fe0f0349816 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+14026774 [ 27] 0x00007fe0f045083e /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+15104062 [ 28] 0x00007fe0f0458c3f /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+15137855 [ 29] 0x00007fe0f03b07b4 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+14448564 [ 30] 0x00007fe0f03d7bad /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+14609325 [ 31] 0x00007fe0efb5a2db /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+05706459 [ 32] 0x00007fe0efb5c514 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+05715220 [ 33] 0x00007fe0efb592bd /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+05702333 [ 34] 0x00007fe0efb46791 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+05625745 [ 35] 0x00007fe0efb469c9 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+05626313 [ 36] 0x00007fe0efb58ac6 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+05700294 [ 37] 0x00007fe0efb58bc6 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+05700550 [ 38] 0x00007fe0efc926e9 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+06985449 [ 39] 0x00007fe0efc95e23 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+06999587 [ 40] 0x00007fe0f0200891 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+12679313 [ 41] 0x00007fe0f01af063 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+12345443 [ 42] 0x00007fe0f01b30af /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+12361903 [ 43] 0x00007fe0f01b60e2 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+12374242 [ 44] 0x00007fe0f024f72f /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+13002543 [ 45] 0x00007fe0f024fa19 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwm_lxe.so+13003289 [ 46] 0x00007fe0f80714c4 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwbridge.so+00341188 _Z8mnParserv+00000596 [ 47] 0x00007fe0f2e6c5b5 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwmcr.so+01017269 [ 48] 0x00007fe10795442b /usr/local/MATLAB/R2019b/bin/glnxa64/libmwmvm.so+03097643 _ZN14cmddistributor15PackagedTaskIIP10invokeFuncIN7mwboost8functionIFvvEEEEENS2_10shared_ptrINS2_13unique_futureIDTclfpEEEEEERKT+00000059 [ 49] 0x00007fe107954518 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwmvm.so+03097880 _ZNSt17_Function_handlerIFN7mwboost3anyEvEZN14cmddistributor15PackagedTaskIIP10createFuncINS0_8functionIFvvEEEEESt8functionIS2_ET_EUlvE_E9_M_invokeERKSt9_Any_data+00000024 [ 50] 0x00007fe0f2fbc89c /usr/local/MATLAB/R2019b/bin/glnxa64/libmwiqm.so+00751772 _ZN7mwboost6detail8function21function_obj_invoker0ISt8functionIFNS_3anyEvEES4_E6invokeERNS1_15function_bufferE+00000028 [ 51] 0x00007fe0f2fbc557 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwiqm.so+00750935 _ZN3iqm18PackagedTaskPlugin7executeEP15inWorkSpace_tag+00000439 [ 52] 0x00007fe0f2e5b015 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwmcr.so+00946197 [ 53] 0x00007fe0f2fa16a0 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwiqm.so+00640672 [ 54] 0x00007fe0f2f85e01 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwiqm.so+00527873 [ 55] 0x00007fe0f2f86a7f /usr/local/MATLAB/R2019b/bin/glnxa64/libmwiqm.so+00531071 [ 56] 0x00007fe0f2e42575 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwmcr.so+00845173 [ 57] 0x00007fe0f2e42b93 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwmcr.so+00846739 [ 58] 0x00007fe0f2e43404 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwmcr.so+00848900 [ 59] 0x00007fe105c31bdd /usr/local/MATLAB/R2019b/bin/glnxa64/libmwboost_thread.so.1.65.1+00080861 [ 60] 0x00007fe106bf66db /lib/x86_64-linux-gnu/libpthread.so.0+00030427 [ 61] 0x00007fe106387a3f /lib/x86_64-linux-gnu/libc.so.6+01186367 clone+00000063 [ 62] 0x0000000000000000 +00000000

This error was detected while a MEX-file was running. If the MEX-file is not an official MathWorks function, please examine its source code for errors. Please consult the External Interfaces Guide for information on debugging MEX-files.

curtcorum commented 4 years ago

Slightly different behavior after cleaning up arrays used to all be single. This probably does not save anything on the gpu (since being loaded as single?) but frees more memory on the cpu side.

curt@green:~/ngfn_simulation_recon/matlab$ nice matlab Gtk-Message: 19:01:55.975: Failed to load module "canberra-gtk-module" out of memory in /home/curt/Dropbox/ngfn_simulation_recon/matlab/gpuNUFFT/CUDA/inc/cuda_utils.hpp at line 40 pure virtual method called Segmentation fault (core dumped)

   Segmentation violation detected at Fri Aug 07 19:43:38 2020 -0500

Configuration: Crash Decoding : Disabled - No sandbox or build area path Crash Mode : continue (default) Default Encoding : UTF-8 Deployed : false Desktop Environment : GNOME-Flashback:GNOME GNU C Library : 2.27 stable Graphics Driver : NVIDIA Corporation Quadro RTX 8000/PCIe/SSE2 Version 4.6.0 NVIDIA 440.100 Graphics card 1 : 0x10de ( 0x10de ) 0x1e30 Version 440.100.0.0 (0-0-0) Graphics card 2 : 0x10de ( 0x10de ) 0x1e30 Version 440.100.0.0 (0-0-0) Java Version : Java 1.8.0_202-b08 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode MATLAB Architecture : glnxa64 MATLAB Entitlement ID : 4651242 MATLAB Root : /usr/local/MATLAB/R2019b MATLAB Version : 9.7.0.1319299 (R2019b) Update 5 OpenGL : hardware Operating System : Ubuntu 18.04.4 LTS Process ID : 105233 Processor ID : x86 Family 143 Model 49 Stepping 0, AuthenticAMD Session Key : 0ddffd23-b786-465c-a283-d2e3cb6cfadf Static TLS mitigation : Enabled: Full Window System : The X.Org Foundation (12005000), display :1

Fault Count: 1

Abnormal termination: Segmentation violation

Register State (from fault): RAX = 00007f9d34484d70 RBX = 00007f9c9c024170 RCX = 00007f9d4cce3f60 RDX = 00007f9f00000001 RSP = 00007f9fa192af08 RBP = 00007f9fa192af30 RSI = 00007f9e21930d28 RDI = 00007f9d34484d70

R8 = 00007f9dfc6ba974 R9 = 00007f9fac05fa90 R10 = 0000000001d94f60 R11 = 00007f9fbdb6e9c0 R12 = 00007f9dfc6ba800 R13 = 00007f9d73cae220 R14 = 00007f9d4ce9b0c0 R15 = 00007f9d342fd600

RIP = 00007f9f00000001 EFL = 0000000000010202

CS = 0033 FS = 0000 GS = 0000

Stack Trace (from fault): [ 0] 0x00007f9f00000001 +00000000 [ 1] 0x00007f9fbdb1a0f1 /lib/x86_64-linux-gnu/libc.so.6+00274673 [ 2] 0x00007f9fa192afb0 +00000000

This error was detected while a MEX-file was running. If the MEX-file is not an official MathWorks function, please examine its source code for errors. Please consult the External Interfaces Guide for information on debugging MEX-files.

      std::terminate() detected at Fri Aug 07 19:43:47 2020 -0500

Configuration: Crash Decoding : Disabled - No sandbox or build area path Crash Mode : continue (default) Default Encoding : UTF-8 Deployed : false Desktop Environment : GNOME-Flashback:GNOME GNU C Library : 2.27 stable Graphics Driver : NVIDIA Corporation Quadro RTX 8000/PCIe/SSE2 Version 4.6.0 NVIDIA 440.100 Graphics card 1 : 0x10de ( 0x10de ) 0x1e30 Version 440.100.0.0 (0-0-0) Graphics card 2 : 0x10de ( 0x10de ) 0x1e30 Version 440.100.0.0 (0-0-0) Java Version : Java 1.8.0_202-b08 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode MATLAB Architecture : glnxa64 MATLAB Entitlement ID : 4651242 MATLAB Root : /usr/local/MATLAB/R2019b MATLAB Version : 9.7.0.1319299 (R2019b) Update 5 OpenGL : hardware Operating System : Ubuntu 18.04.4 LTS Process ID : 105233 Processor ID : x86 Family 143 Model 49 Stepping 0, AuthenticAMD Session Key : 0ddffd23-b786-465c-a283-d2e3cb6cfadf Static TLS mitigation : Enabled: Full Window System : The X.Org Foundation (12005000), display :1

Fault Count: 2

Abnormal termination: Segmentation violation

Register State (from fault): RAX = 00007f9d34484d70 RBX = 00007f9c9c024170 RCX = 00007f9d4cce3f60 RDX = 00007f9f00000001 RSP = 00007f9fa192af08 RBP = 00007f9fa192af30 RSI = 00007f9e21930d28 RDI = 00007f9d34484d70

R8 = 00007f9dfc6ba974 R9 = 00007f9fac05fa90 R10 = 0000000001d94f60 R11 = 00007f9fbdb6e9c0 R12 = 00007f9dfc6ba800 R13 = 00007f9d73cae220 R14 = 00007f9d4ce9b0c0 R15 = 00007f9d342fd600

RIP = 00007f9f00000001 EFL = 0000000000010202

CS = 0033 FS = 0000 GS = 0000

Stack Trace (from fault): [ 0] 0x00007f9f00000001 +00000000 [ 1] 0x00007f9fbdb1a0f1 /lib/x86_64-linux-gnu/libc.so.6+00274673 [ 2] 0x00007f9fa192afb0 +00000000

Abnormal termination: std::terminate()

Register State (captured): RAX = 00007f9ebf0a3000 RBX = 00007f9fbc0eed98 RCX = 000000000000000a RDX = 00007f9fbc0cfe00 RSP = 00007f9ebf0a2400 RBP = 00007f9ebf0a27c0 RSI = 00007f9fbc0972e6 RDI = 00007f9ebf0a2410

R8 = 000000000000ffff R9 = 29286574616e696d R10 = 00000000bf3fe958 R11 = 0000000000000000 R12 = 00007f9fbc0ded08 R13 = 00007f9ebf0a2b80 R14 = 00007f9ebf0a3070 R15 = 00007f9ebf0a3040

RIP = 00007f9fbbfe70be EFL = 00007f9fbc1bcbc0

CS = 001a FS = 0000 GS = 0000

Stack Trace (captured): [ 0] 0x00007f9fbbfde623 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwfl.so+00828963 [ 1] 0x00007f9fbbfde81c /usr/local/MATLAB/R2019b/bin/glnxa64/libmwfl.so+00829468 _ZN10foundation4core4diag15stacktrace_base7captureEm+00000028 [ 2] 0x00007f9fbbfe0f62 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwfl.so+00839522 [ 3] 0x00007f9faa759843 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwmcr.so+01218627 [ 4] 0x00007f9faa75c447 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwmcr.so+01229895 [ 5] 0x00007f9faa75c67a /usr/local/MATLAB/R2019b/bin/glnxa64/libmwmcr.so+01230458 [ 6] 0x00007f9faa75cc41 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwmcr.so+01231937 [ 7] 0x00007f9faa75dbc7 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwmcr.so+01235911 [ 8] 0x00007f9fbe16d3a6 /usr/local/MATLAB/R2019b/bin/glnxa64/../../sys/os/glnxa64/libstdc++.so.6+00582566 [ 9] 0x00007f9fbe16d3f1 /usr/local/MATLAB/R2019b/bin/glnxa64/../../sys/os/glnxa64/libstdc++.so.6+00582641 [ 10] 0x00007f9fbe16debf /usr/local/MATLAB/R2019b/bin/glnxa64/../../sys/os/glnxa64/libstdc++.so.6+00585407 [ 11] 0x00007f9fbc512e26 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwms.so+02113062 _ZNK10foundation7msg_svc8eventmgr9BaseEvent13getFilterTagsB5cxx11Ev+00000022 [ 12] 0x00007f9fbc5567cc /usr/local/MATLAB/R2019b/bin/glnxa64/libmwms.so+02389964 [ 13] 0x00007f9fbc557048 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwms.so+02392136 [ 14] 0x00007f9fbc5735cb /usr/local/MATLAB/R2019b/bin/glnxa64/libmwms.so+02508235 [ 15] 0x00007f9fbc577716 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwms.so+02524950 [ 16] 0x00007f9fbc577a0c /usr/local/MATLAB/R2019b/bin/glnxa64/libmwms.so+02525708 [ 17] 0x00007f9fbc577367 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwms.so+02524007 [ 18] 0x00007f9fbc55333c /usr/local/MATLAB/R2019b/bin/glnxa64/libmwms.so+02376508 [ 19] 0x00007f9fbc551f31 /usr/local/MATLAB/R2019b/bin/glnxa64/libmwms.so+02371377 [ 20] 0x00007f9fbd4a2bdd /usr/local/MATLAB/R2019b/bin/glnxa64/libmwboost_thread.so.1.65.1+00080861 [ 21] 0x00007f9fbe4676db /lib/x86_64-linux-gnu/libpthread.so.0+00030427 [ 22] 0x00007f9fbdbf8a3f /lib/x86_64-linux-gnu/libc.so.6+01186367 clone+00000063 [ 23] 0x0000000000000000 +00000000