search

microsoft / CNTK

Microsoft Cognitive Toolkit (CNTK), an open source deep-learning toolkit
https://docs.microsoft.com/cognitive-toolkit/
Other
17.5k stars 4.29k forks source link

CNTK Linux Crash backtrace and memory dump on ONNX load #3353

Open tblattner opened 6 years ago

tblattner commented 6 years ago

I am trying to get CNTK operating in my C++ workflow, and have begun the move to use ONNX.

I use this code to load the ONNX file: CNTK::FunctionPtr model = CNTK::Function::Load(wcntkModelFile, CNTK::DeviceDescriptor::GPUDevice(0));

This works fine on cntk model files that was trained using CNTK. I have another model that was created from Caffe2, and then converted into an onnx model file. I try the same code as above to load the onnx model, but get the error (shown in the details below).

I've tried the binary GPU release for CNTK -2-5-1, as well as building CNTK from source. In both cases it produces the dump, whereas the CNTK-generated model file will work fine.

Any ideas on how to resolve this issue? If need be I can send the onnx file that I am using.

Error:

Error in `/home/tjb3/work/MITS/src/HTGS-MITS/cmake-build-debug/mits_exec': free(): invalid pointer: 0x00007fddd40141a8 ======= Backtrace: ========= /lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7fdf6bcbd7e5] /lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7fdf6bcc637a] /lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7fdf6bcca53c] /home/tjb3/apps/cntk/build/release/lib/libCntk.Core-2.5.1.so(_ZN4CNTK5proto10Dictionary27MergePartialFromCodedStreamEPN6google8protobuf2io16CodedInputStreamE+0x19e)[0x7fdf6e86ee3e] /home/tjb3/apps/cntk/build/release/lib/libCntk.Core-2.5.1.so(_ZN6google8protobuf11MessageLite20ParseFromCodedStreamEPNS0_2io16CodedInputStreamE+0x96)[0x7fdf6ef4b74e] /home/tjb3/apps/cntk/build/release/lib/libCntk.Core-2.5.1.so(_ZN4CNTK12ParseMessageERN6google8protobuf2io19ZeroCopyInputStreamERNS1_7MessageE+0x12e)[0x7fdf6ebf18ae] /home/tjb3/apps/cntk/build/release/lib/libCntk.Core-2.5.1.so(_ZN4CNTK10Serializer4ReadERSiRKSt8functionIFbRN6google8protobuf2io19ZeroCopyInputStreamEEE+0x40)[0x7fdf6ebf7bf0] /home/tjb3/apps/cntk/build/release/lib/libCntk.Core-2.5.1.so(_ZN4CNTK10Serializer4ReadERSiRNS_10DictionaryE+0x97)[0x7fdf6ebf7ce7] /home/tjb3/apps/cntk/build/release/lib/libCntk.Core-2.5.1.so(_ZN4CNTKrsERSiRNS_10DictionaryE+0x3d)[0x7fdf6ebf7dcd] /home/tjb3/apps/cntk/build/release/lib/libCntk.Core-2.5.1.so(_ZN4CNTK8Function4LoadERKNSt7__cxx1112basic_stringIwSt11char_traitsIwESaIwEEERKNS_16DeviceDescriptorENS_11ModelFormatE+0xa3)[0x7fdf6ea130d3] /home/tjb3/work/MITS/src/HTGS-MITS/cmake-build-debug/mits_exec(_ZN17StarInferenceTask17initializeCudaGPUEv+0x1f0)[0x65017e] /home/tjb3/work/MITS/src/HTGS-MITS/cmake-build-debug/mits_exec(_ZN4htgs9ICudaTaskI10TensorData17ProcessedMetaDataE10initializeEv+0x1d7)[0x65fa15] /home/tjb3/work/MITS/src/HTGS-MITS/cmake-build-debug/mits_exec(_ZN4htgs8AnyITask10initializeEmm+0x43)[0x6c6e8d] /home/tjb3/work/MITS/src/HTGS-MITS/cmake-build-debug/mits_exec(_ZN4htgs5ITaskI10TensorData17ProcessedMetaDataE10initializeEmmPNS_11TaskManagerIS1_S2_EE+0x3b)[0x759b13] /home/tjb3/work/MITS/src/HTGS-MITS/cmake-build-debug/mits_exec(_ZN4htgs11TaskManagerI10TensorData17ProcessedMetaDataE10initializeEv+0x57)[0x740765] /home/tjb3/work/MITS/src/HTGS-MITS/cmake-build-debug/mits_exec(_ZN4htgs17TaskManagerThread3runEv+0x3a)[0x6c867a] /home/tjb3/work/MITS/src/HTGS-MITS/cmake-build-debug/mits_exec(_ZNKSt12_Mem_fn_baseIMN4htgs17TaskManagerThreadEFivELb1EEclIJEvEEiPS1DpOT+0x65)[0x768a2b] /home/tjb3/work/MITS/src/HTGS-MITS/cmake-build-debug/mits_exec(_ZNSt12_Bind_simpleIFSt7_Mem_fnIMN4htgs17TaskManagerThreadEFivEEPS2_EE9_M_invokeIILm0EEEEiSt12_Index_tupleIIXspT_EEE+0x43)[0x763227] /home/tjb3/work/MITS/src/HTGS-MITS/cmake-build-debug/mits_exec(_ZNSt12_Bind_simpleIFSt7_Mem_fnIMN4htgs17TaskManagerThreadEFivEEPS2_EEclEv+0x2c)[0x75afb0] /home/tjb3/work/MITS/src/HTGS-MITS/cmake-build-debug/mits_exec(_ZNSt6thread5_ImplISt12_Bind_simpleIFSt7_Mem_fnIMN4htgs17TaskManagerThreadEFivEEPS4_EEE6_M_runEv+0x1c)[0x7442e0] /usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xb8c80)[0x7fdf6c2dec80] /lib/x86_64-linux-gnu/libpthread.so.0(+0x76ba)[0x7fdf6c8b86ba] /lib/x86_64-linux-gnu/libc.so.6(clone+0x6d)[0x7fdf6bd4d41d] ======= Memory map: ======== 00400000-008a7000 r-xp 00000000 08:11 23594845 /home/tjb3/work/MITS/src/HTGS-MITS/cmake-build-debug/mits_exec 00aa6000-00aa9000 r--p 004a6000 08:11 23594845 /home/tjb3/work/MITS/src/HTGS-MITS/cmake-build-debug/mits_exec 00aa9000-00aab000 rw-p 004a9000 08:11 23594845 /home/tjb3/work/MITS/src/HTGS-MITS/cmake-build-debug/mits_exec 00aab000-00aac000 rw-p 00000000 00:00 0 00c8a000-175bf000 rw-p 00000000 00:00 0 [heap] 200000000-200200000 rw-s 00000000 00:06 514 /dev/nvidiactl 200200000-200400000 ---p 00000000 00:00 0 200400000-200404000 rw-s 00000000 00:06 514 /dev/nvidiactl 200404000-200600000 ---p 00000000 00:00 0 200600000-200a00000 rw-s 00000000 00:06 514 /dev/nvidiactl 200a00000-201600000 ---p 00000000 00:00 0 201600000-201604000 rw-s 00000000 00:06 514 /dev/nvidiactl 201604000-201800000 ---p 00000000 00:00 0 201800000-201c00000 rw-s 00000000 00:06 514 /dev/nvidiactl 201c00000-201c04000 rw-s 00000000 00:06 514 /dev/nvidiactl 201c04000-201e00000 ---p 00000000 00:00 0 201e00000-202200000 rw-s 00000000 00:06 514 /dev/nvidiactl 202200000-202204000 rw-s 00000000 00:06 514 /dev/nvidiactl 202204000-202400000 ---p 00000000 00:00 0 202400000-202800000 rw-s 00000000 00:06 514 /dev/nvidiactl 202800000-202804000 rw-s 00000000 00:06 514 /dev/nvidiactl 202804000-202a00000 ---p 00000000 00:00 0 202a00000-202e00000 rw-s 00000000 00:06 514 /dev/nvidiactl 202e00000-202e04000 rw-s 00000000 00:06 514 /dev/nvidiactl 202e04000-203000000 ---p 00000000 00:00 0 203000000-203400000 rw-s 00000000 00:06 514 /dev/nvidiactl 203400000-203404000 rw-s 00000000 00:06 514 /dev/nvidiactl 203404000-203600000 ---p 00000000 00:00 0 203600000-203a00000 rw-s 00000000 00:06 514 /dev/nvidiactl 203a00000-203a04000 rw-s 00000000 00:06 514 /dev/nvidiactl 203a04000-203c00000 ---p 00000000 00:00 0 203c00000-204000000 rw-s 00000000 00:06 514 /dev/nvidiactl 204000000-204004000 rw-s 00000000 00:06 514 /dev/nvidiactl 204004000-204200000 ---p 00000000 00:00 0 204200000-204600000 rw-s 00000000 00:06 514 /dev/nvidiactl 204600000-204604000 rw-s 00000000 00:06 514 /dev/nvidiactl 204604000-204800000 ---p 00000000 00:00 0 204800000-204c00000 rw-s 00000000 00:06 514 /dev/nvidiactl 204c00000-204c04000 rw-s 00000000 00:06 514 /dev/nvidiactl 204c04000-204e00000 ---p 00000000 00:00 0 204e00000-205200000 rw-s 00000000 00:06 514 /dev/nvidiactl 205200000-205204000 rw-s 00000000 00:06 514 /dev/nvidiactl 205204000-205400000 ---p 00000000 00:00 0 205400000-205800000 rw-s 00000000 00:06 514 /dev/nvidiactl 205800000-205804000 rw-s 00000000 00:06 514 /dev/nvidiactl 205804000-205a00000 ---p 00000000 00:00 0 205a00000-205e00000 rw-s 00000000 00:06 514 /dev/nvidiactl 205e00000-205e04000 rw-s 00000000 00:06 514 /dev/nvidiactl 205e04000-206000000 ---p 00000000 00:00 0 206000000-206400000 rw-s 00000000 00:06 514 /dev/nvidiactl 206400000-206404000 rw-s 00000000 00:06 514 /dev/nvidiactl 206404000-206600000 ---p 00000000 00:00 0 206600000-206a00000 rw-s 00000000 00:06 514 /dev/nvidiactl 206a00000-206a04000 rw-s 00000000 00:06 514 /dev/nvidiactl 206a04000-206c00000 ---p 00000000 00:00 0 206c00000-207000000 rw-s 00000000 00:06 514 /dev/nvidiactl 207000000-207200000 ---p 00000000 00:00 0 207200000-207400000 rw-s 00000000 00:06 514 /dev/nvidiactl 207400000-300200000 ---p 00000000 00:00 0 10000000000-10204000000 ---p 00000000 00:00 0 7fdd6c000000-7fdd6c121000 rw-p 00000000 00:00 0 7fdd6c121000-7fdd70000000 ---p 00000000 00:00 0 7fdd74000000-7fdd74021000 rw-p 00000000 00:00 0 7fdd74021000-7fdd78000000 ---p 00000000 00:00 0 7fdd78000000-7fdd78305000 rw-p 00000000 00:00 0 7fdd78305000-7fdd7c000000 ---p 00000000 00:00 0 7fdd7c000000-7fddcc000000 ---p 00000000 00:00 0 7fddcc000000-7fddcc021000 rw-p 00000000 00:00 0 7fddcc021000-7fddd0000000 ---p 00000000 00:00 0 7fddd0000000-7fddd0580000 rw-p 00000000 00:00 0 7fddd0580000-7fddd4000000 ---p 00000000 00:00 0 7fddd4000000-7fddd4021000 rw-p 00000000 00:00 0 7fddd4021000-7fddd8000000 ---p 00000000 00:00 0 7fddd8000000-7fddd8021000 rw-p 00000000 00:00 0 7fddd8021000-7fdddc000000 ---p 00000000 00:00 0 7fdddc000000-7fdddc02d000 rw-p 00000000 00:00 0 7fdddc02d000-7fdde0000000 ---p 00000000 00:00 0 7fdde0000000-7fdde0021000 rw-p 00000000 00:00 0 7fdde0021000-7fdde4000000 ---p 00000000 00:00 0 7fdde77f7000-7fdde77f8000 ---p 00000000 00:00 0 7fdde77f8000-7fdde7ff8000 rw-p 00000000 00:00 0 7fdde7ff8000-7fdde7ff9000 ---p 00000000 00:00 0 7fdde7ff9000-7fdde87f9000 rw-p 00000000 00:00 0 7fdde87f9000-7fdde87fa000 ---p 00000000 00:00 0 7fdde87fa000-7fdde8ffa000 rw-p 00000000 00:00 0 7fdde8ffa000-7fdde8ffb000 ---p 00000000 00:00 0 7fdde8ffb000-7fdde97fb000 rw-p 00000000 00:00 0 7fdde97fb000-7fdde97fc000 ---p 00000000 00:00 0 7fdde97fc000-7fdde9ffc000 rw-p 00000000 00:00 0 7fdde9ffc000-7fdde9ffd000 ---p 00000000 00:00 0 7fdde9ffd000-7fddea7fd000 rw-p 00000000 00:00 0 7fddea7fd000-7fddea7fe000 ---p 00000000 00:00 0 7fddea7fe000-7fddeaffe000 rw-p 00000000 00:00 0 7fddeaffe000-7fddeafff000 ---p 00000000 00:00 0 7fddeafff000-7fddeb7ff000 rw-p 00000000 00:00 0 7fddeb7ff000-7fddeb800000 ---p 00000000 00:00 0 7fddeb800000-7fddec000000 rw-p 00000000 00:00 0 7fddec000000-7fddec021000 rw-p 00000000 00:00 0 7fddec021000-7fddf0000000 ---p 00000000 00:00 0 7fddf07f9000-7fddf07fa000 ---p 00000000 00:00 0 7fddf07fa000-7fddf0ffa000 rw-p 00000000 00:00 0 7fddf0ffa000-7fddf0ffb000 ---p 00000000 00:00 0 7fddf0ffb000-7fddf17fb000 rw-p 00000000 00:00 0 7fddf17fb000-7fddf17fc000 ---p 00000000 00:00 0 7fddf17fc000-7fddf1ffc000 rw-p 00000000 00:00 0 7fddf1ffc000-7fddf1ffd000 ---p 00000000 00:00 0 7fddf1ffd000-7fddf27fd000 rw-p 00000000 00:00 0 7fddf27fd000-7fddf27fe000 ---p 00000000 00:00 0 7fddf27fe000-7fddf2ffe000 rw-p 00000000 00:00 0 7fddf2ffe000-7fddf2fff000 ---p 00000000 00:00 0 7fddf2fff000-7fddf37ff000 rw-p 00000000 00:00 0 7fddf37ff000-7fddf3800000 ---p 00000000 00:00 0 7fddf3800000-7fddf4000000 rw-p 00000000 00:00 0 7fddf4000000-7fddf4101000 rw-p 00000000 00:00 0 7fddf4101000-7fddf8000000 ---p 00000000 00:00 0 7fddf87f9000-7fddf87fa000 ---p 00000000 00:00 0 7fddf87fa000-7fddf8ffa000 rw-p 00000000 00:00 0 7fddf8ffa000-7fddf8ffb000 ---p 00000000 00:00 0 7fddf8ffb000-7fddf97fb000 rw-p 00000000 00:00 0 7fddf97fb000-7fddf97fc000 ---p 00000000 00:00 0 7fddf97fc000-7fddf9ffc000 rw-p 00000000 00:00 0 7fddf9ffc000-7fddf9ffd000 ---p 00000000 00:00 0 7fddf9ffd000-7fddfa7fd000 rw-p 00000000 00:00 0 7fddfa7fd000-7fddfa7fe000 ---p 00000000 00:00 0 7fddfa7fe000-7fddfaffe000 rw-p 00000000 00:00 0 7fddfaffe000-7fddfafff000 ---p 00000000 00:00 0 7fddfafff000-7fddfb7ff000 rw-p 00000000 00:00 0 7fddfb7ff000-7fddfb800000 ---p 00000000 00:00 0 7fddfb800000-7fddfc000000 rw-p 00000000 00:00 0 7fddfc000000-7fddfc201000 rw-p 00000000 00:00 0 7fddfc201000-7fde00000000 ---p 00000000 00:00 0 7fde007f1000-7fde007f2000 ---p 00000000 00:00 0 7fde007f2000-7fde00ff2000 rw-p 00000000 00:00 0 7fde00ff2000-7fde00ff3000 ---p 00000000 00:00 0 7fde00ff3000-7fde017f3000 rw-p 00000000 00:00 0 7fde017f3000-7fde017f4000 ---p 00000000 00:00 0 7fde017f4000-7fde01ff4000 rw-p 00000000 00:00 0 7fde01ff4000-7fde01ff5000 ---p 00000000 00:00 0 7fde01ff5000-7fde027f5000 rw-p 00000000 00:00 0 7fde027f5000-7fde027f6000 ---p 00000000 00:00 0 7fde027f6000-7fde02ff6000 rw-p 00000000 00:00 0 7fde02ff6000-7fde02ff7000 ---p 00000000 00:00 0 7fde02ff7000-7fde037f7000 rw-p 00000000 00:00 0 7fde037f7000-7fde037f8000 ---p 00000000 00:00 0 7fde037f8000-7fde03ff8000 rw-p 00000000 00:00 0 7fde03ff8000-7fde03ff9000 ---p 00000000 00:00 0 7fde03ff9000-7fde047f9000 rw-p 00000000 00:00 0 7fde047f9000-7fde047fa000 ---p 00000000 00:00 0 7fde047fa000-7fde04ffa000 rw-p 00000000 00:00 0 7fde04ffa000-7fde04ffb000 ---p 00000000 00:00 0 7fde04ffb000-7fde057fb000 rw-p 00000000 00:00 0 7fde057fb000-7fde057fc000 ---p 00000000 00:00 0 7fde057fc000-7fde05ffc000 rw-p 00000000 00:00 0 7fde05ffc000-7fde05ffd000 ---p 00000000 00:00 0 7fde05ffd000-7fde067fd000 rw-p 00000000 00:00 0 7fde067fd000-7fde067fe000 ---p 00000000 00:00 0 7fde067fe000-7fde06ffe000 rw-p 00000000 00:00 0 7fde06ffe000-7fde06fff000 ---p 00000000 00:00 0 7fde06fff000-7fde077ff000 rw-p 00000000 00:00 0 7fde077ff000-7fde07800000 ---p 00000000 00:00 0 7fde07800000-7fde08000000 rw-p 00000000 00:00 0 7fde08000000-7fde08602000 rw-p 00000000 00:00 0 7fde08602000-7fde0c000000 ---p 00000000 00:00 0 7fde0c7f9000-7fde0c7fa000 ---p 00000000 00:00 0 7fde0c7fa000-7fde0cffa000 rw-p 00000000 00:00 0 7fde0cffa000-7fde0cffb000 ---p 00000000 00:00 0 7fde0cffb000-7fde0d7fb000 rw-p 00000000 00:00 0 7fde0d7fb000-7fde0d7fc000 ---p 00000000 00:00 0 7fde0d7fc000-7fde0dffc000 rw-p 00000000 00:00 0 7fde0dffc000-7fde0dffd000 ---p 00000000 00:00 0 7fde0dffd000-7fde0e7fd000 rw-p 00000000 00:00 0 7fde0e7fd000-7fde0e7fe000 ---p 00000000 00:00 0 7fde0e7fe000-7fde0effe000 rw-p 00000000 00:00 0 7fde0effe000-7fde0efff000 ---p 00000000 00:00 0 7fde0efff000-7fde0f7ff000 rw-p 00000000 00:00 0 7fde0f7ff000-7fde0f800000 ---p 00000000 00:00 0 7fde0f800000-7fde10000000 rw-p 00000000 00:00 0 7fde10000000-7fde11203000 rw-p 00000000 00:00 0 7fde11203000-7fde14000000 ---p 00000000 00:00 0 7fde147f9000-7fde147fa000 ---p 00000000 00:00 0 7fde147fa000-7fde14ffa000 rw-p 00000000 00:00 0 7fde14ffa000-7fde14ffb000 ---p 00000000 00:00 0 7fde14ffb000-7fde157fb000 rw-p 00000000 00:00 0 7fde157fb000-7fde157fc000 ---p 00000000 00:00 0 7fde157fc000-7fde15ffc000 rw-p 00000000 00:00 0 7fde15ffc000-7fde15ffd000 ---p 00000000 00:00 0 7fde15ffd000-7fde167fd000 rw-p 00000000 00:00 0 7fde167fd000-7fde167fe000 ---p 00000000 00:00 0 7fde167fe000-7fde16ffe000 rw-p 00000000 00:00 0 7fde16ffe000-7fde16fff000 ---p 00000000 00:00 0 7fde16fff000-7fde177ff000 rw-p 00000000 00:00 0 7fde177ff000-7fde17800000 ---p 00000000 00:00 0 7fde17800000-7fde18000000 rw-p 00000000 00:00 0 7fde18000000-7fde1b206000 rw-p 00000000 00:00 0 7fde1b206000-7fde1c000000 ---p 00000000 00:00 0 7fde1c7c9000-7fde1c7ca000 ---p 00000000 00:00 0 7fde1c7ca000-7fde1cfca000 rw-p 00000000 00:00 0 7fde1cfca000-7fde1cfcb000 ---p 00000000 00:00 0 7fde1cfcb000-7fde1d7cb000 rw-p 00000000 00:00 0 7fde1d7cb000-7fde1d7cc000 ---p 00000000 00:00 0 7fde1d7cc000-7fde1dfcc000 rw-p 00000000 00:00 0 7fde1dfcc000-7fde1dfcd000 ---p 00000000 00:00 0 7fde1dfcd000-7fde1e7cd000 rw-p 00000000 00:00 0 7fde1e7cd000-7fde1e7ce000 ---p 00000000 00:00 0 7fde1e7ce000-7fde1efce000 rw-p 00000000 00:00 0 7fde1efce000-7fde1efcf000 ---p 00000000 00:00 0 7fde1efcf000-7fde1f7cf000 rw-p 00000000 00:00 0 7fde1f7ff000-7fde1f800000 ---p 00000000 00:00 0 7fde1f800000-7fde20000000 rw-p 00000000 00:00 0 7fde20000000-7fde23206000 rw-p 00000000 00:00 0 7fde23206000-7fde24000000 ---p 00000000 00:00 0 7fde247c9000-7fde247ca000 ---p 00000000 00:00 0 7fde247ca000-7fde24fca000 rw-p 00000000 00:00 0 7fde24fca000-7fde24fcb000 ---p 00000000 00:00 0 7fde24fcb000-7fde257cb000 rw-p 00000000 00:00 0 7fde257cb000-7fde257cc000 ---p 00000000 00:00 0 7fde257cc000-7fde25fcc000 rw-p 00000000 00:00 0 7fde25fcc000-7fde25fcd000 ---p 00000000 00:00 0 7fde25fcd000-7fde267cd000 rw-p 00000000 00:00 0 7fde267cd000-7fde267ce000 ---p 00000000 00:00 0 7fde267ce000-7fde26fce000 rw-p 00000000 00:00 0 7fde26fce000-7fde26fcf000 ---p 00000000 00:00 0 7fde26fcf000-7fde277cf000 rw-p 00000000 00:00 0 7fde277cf000-7fde277d0000 ---p 00000000 00:00 0 7fde277d0000-7fde27fd0000 rw-p 00000000 00:00 0 7fde27fd0000-7fde27fd1000 ---p 00000000 00:00 0 7fde27fd1000-7fde287d1000 rw-p 00000000 00:00 0 7fde287d1000-7fde287d2000 ---p 00000000 00:00 0 7fde287d2000-7fde28fd2000 rw-p 00000000 00:00 0 7fde28ffa000-7fde28ffb000 ---p 00000000 00:00 0 7fde28ffb000-7fde297fb000 rw-p 00000000 00:00 0 7fde297fb000-7fde297fc000 ---p 00000000 00:00 0 7fde297fc000-7fde29ffc000 rw-p 00000000 00:00 0 7fde29ffc000-7fde29ffd000 ---p 00000000 00:00 0 7fde29ffd000-7fde2a7fd000 rw-p 00000000 00:00 0 7fde2a7fd000-7fde2a7fe000 ---p 00000000 00:00 0 7fde2a7fe000-7fde2affe000 rw-p 00000000 00:00 0 7fde2affe000-7fde2afff000 ---p 00000000 00:00 0 7fde2afff000-7fde2b7ff000 rw-p 00000000 00:00 0 7fde2b7ff000-7fde2b800000 ---p 00000000 00:00 0 7fde2b800000-7fde2c000000 rw-p 00000000 00:00 0 7fde2c000000-7fde2f206000 rw-p 00000000 00:00 0 7fde2f206000-7fde30000000 ---p 00000000 00:00 0 7fde307d1000-7fde307d2000 ---p 00000000 00:00 0 7fde307d2000-7fde30fd2000 rw-p 00000000 00:00 0 7fde30fd2000-7fde30fd3000 ---p 00000000 00:00 0 7fde30fd3000-7fde317d3000 rw-p 00000000 00:00 0 7fde317d3000-7fde317d4000 ---p 00000000 00:00 0 7fde317d4000-7fde31fd4000 rw-p 00000000 00:00 0 7fde31fd4000-7fde31fd5000 ---p 00000000 00:00 0 7fde31fd5000-7fde327d5000 rw-p 00000000 00:00 0 7fde327fd000-7fde327fe000 ---p 00000000 00:00 0 7fde327fe000-7fde32ffe000 rw-p 00000000 00:00 0 7fde32ffe000-7fde32fff000 ---p 00000000 00:00 0 7fde32fff000-7fde337ff000 rw-p 00000000 00:00 0 7fde337ff000-7fde33800000 ---p 00000000 00:00 0 7fde33800000-7fde34000000 rw-p 00000000 00:00 0 7fde34000000-7fde37206000 rw-p 00000000 00:00 0 7fde37206000-7fde38000000 ---p 00000000 00:00 0 7fde387d1000-7fde387d2000 ---p 00000000 00:00 0 7fde387d2000-7fde38fd2000 rw-p 00000000 00:00 0 7fde38fd2000-7fde38fd3000 ---p 00000000 00:00 0 7fde38fd3000-7fde397d3000 rw-p 00000000 00:00 0 7fde397d3000-7fde397d4000 ---p 00000000 00:00 0 7fde397d4000-7fde39fd4000 rw-p 00000000 00:00 0 7fde39fd4000-7fde39fd5000 ---p 00000000 00:00 0 7fde39fd5000-7fde3a7d5000 rw-p 00000000 00:00 0 7fde3a7d5000-7fde3a7d6000 ---p 00000000 00:00 0 7fde3a7d6000-7fde3afd6000 rw-p 00000000 00:00 0 7fde3afd6000-7fde3afd7000 ---p 00000000 00:00 0 7fde3afd7000-7fde3b7d7000 rw-p 00000000 00:00 0 7fde3b7d7000-7fde3b7d8000 ---p 00000000 00:00 0 7fde3b7d8000-7fde3bfd8000 rw-p 00000000 00:00 0 7fde3bfd8000-7fde3bfd9000 ---p 00000000 00:00 0 7fde3bfd9000-7fde3c7d9000 rw-p 00000000 00:00 0 7fde3c7d9000-7fde3c7da000 ---p 00000000 00:00 0 7fde3c7da000-7fde3cfda000 rw-p 00000000 00:00 0 7fde3cfda000-7fde3cfdb000 ---p 00000000 00:00 0 7fde3cfdb000-7fde3d7db000 rw-p 00000000 00:00 0 7fde3d7db000-7fde3d7dc000 ---p 00000000 00:00 0 7fde3d7dc000-7fde3dfdc000 rw-p 00000000 00:00 0 7fde3dfdc000-7fde3dfdd000 ---p 00000000 00:00 0 7fde3dfdd000-7fde3e7dd000 rw-p 00000000 00:00 0 7fde3e7dd000-7fde3e7de000 ---p 00000000 00:00 0 7fde3e7de000-7fde3efde000 rw-p 00000000 00:00 0 7fde3efde000-7fde3efdf000 ---p 00000000 00:00 0 7fde3efdf000-7fde3f7df000 rw-p 00000000 00:00 0 7fde3f7df000-7fde3f7e0000 ---p 00000000 00:00 0 7fde3f7e0000-7fde3ffe0000 rw-p 00000000 00:00 0 7fde3ffe0000-7fde3ffe1000 ---p 00000000 00:00 0 7fde3ffe1000-7fde407e1000 rw-p 00000000 00:00 0 7fde407e1000-7fde407e2000 ---p 00000000 00:00 0 7fde407e2000-7fde40fe2000 rw-p 00000000 00:00 0 7fde40fe2000-7fde40fe3000 ---p 00000000 00:00 0 7fde40fe3000-7fde417e3000 rw-p 00000000 00:00 0 7fde417e3000-7fde417e4000 ---p 00000000 00:00 0 7fde417e4000-7fde41fe4000 rw-p 00000000 00:00 0 7fde41fe4000-7fde41fe5000 ---p 00000000 00:00 0 7fde41fe5000-7fde427e5000 rw-p 00000000 00:00 0 7fde427e5000-7fde427e6000 ---p 00000000 00:00 0 7fde427e6000-7fde42fe6000 rw-p 00000000 00:00 0 7fde42fe6000-7fde42fe7000 ---p 00000000 00:00 0 7fde42fe7000-7fde437e7000 rw-p 00000000 00:00 0 7fde437e7000-7fde437e8000 ---p 00000000 00:00 0 7fde437e8000-7fde43fe8000 rw-p 00000000 00:00 0 7fde43fe8000-7fde43fe9000 ---p 00000000 00:00 0 7fde43fe9000-7fde447e9000 rw-p 00000000 00:00 0 7fde447e9000-7fde447ea000 ---p 00000000 00:00 0 7fde447ea000-7fde44fea000 rw-p 00000000 00:00 0 7fde44fea000-7fde44feb000 ---p 00000000 00:00 0 7fde44feb000-7fde457eb000 rw-p 00000000 00:00 0 7fde457eb000-7fde457ec000 ---p 00000000 00:00 0 7fde457ec000-7fde45fec000 rw-p 00000000 00:00 0 7fde45fec000-7fde45fed000 ---p 00000000 00:00 0 7fde45fed000-7fde467ed000 rw-p 00000000 00:00 0 7fde467ed000-7fde467ee000 ---p 00000000 00:00 0 7fde467ee000-7fde46fee000 rw-p 00000000 00:00 0 7fde46fee000-7fde46fef000 ---p 00000000 00:00 0 7fde46fef000-7fde477ef000 rw-p 00000000 00:00 0 7fde477ef000-7fde477f0000 ---p 00000000 00:00 0 7fde477f0000-7fde47ff0000 rw-p 00000000 00:00 0 7fde47ff0000-7fde47ff1000 ---p 00000000 00:00 0 7fde47ff1000-7fde487f1000 rw-p 00000000 00:00 0 7fde487f1000-7fde487f2000 ---p 00000000 00:00 0 7fde487f2000-7fde48ff2000 rw-p 00000000 00:00 0 7fde48ff2000-7fde48ff3000 ---p 00000000 00:00 0 7fde48ff3000-7fde497f3000 rw-p 00000000 00:00 0 7fde497f3000-7fde497f4000 ---p 00000000 00:00 0 7fde497f4000-7fde49ff4000 rw-p 00000000 00:00 0 7fde49ff4000-7fde49ff5000 ---p 00000000 00:00 0 7fde49ff5000-7fde4a7f5000 rw-p 00000000 00:00 0 7fde4a7f5000-7fde4a7f6000 ---p 00000000 00:00 0 7fde4a7f6000-7fde4aff6000 rw-p 00000000 00:00 0 7fde4aff6000-7fde4aff7000 ---p 00000000 00:00 0 7fde4aff7000-7fde4b7f7000 rw-p 00000000 00:00 0 7fde4b7f7000-7fde4b7f8000 ---p 00000000 00:00 0 7fde4b7f8000-7fde4bff8000 rw-p 00000000 00:00 0 7fde4bff8000-7fde4bff9000 ---p 00000000 00:00 0 7fde4bff9000-7fde4c7f9000 rw-p 00000000 00:00 0 7fde4c7f9000-7fde4c7fa000 ---p 00000000 00:00 0 7fde4c7fa000-7fde4cffa000 rw-p 00000000 00:00 0 7fde4cffa000-7fde4cffb000 ---p 00000000 00:00 0 7fde4cffb000-7fde4d7fb000 rw-p 00000000 00:00 0 7fde4d7fb000-7fde4d7fc000 ---p 00000000 00:00 0 7fde4d7fc000-7fde4dffc000 rw-p 00000000 00:00 0 7fde4dffc000-7fde4dffd000 ---p 00000000 00:00 0 7fde4dffd000-7fde4e7fd000 rw-p 00000000 00:00 0 7fde4e7fd000-7fde4e7fe000 ---p 00000000 00:00 0 7fde4e7fe000-7fde4effe000 rw-p 00000000 00:00 0 7fde4effe000-7fde4efff000 ---p 00000000 00:00 0 7fde4efff000-7fde4f7ff000 rw-p 00000000 00:00 0 7fde4f7ff000-7fde4f800000 ---p 00000000 00:00 0 7fde4f800000-7fde50000000 rw-p 00000000 00:00 0 7fde50000000-7fde58000000 ---p 00000000 00:00 0 7fde58000000-7fde58021000 rw-p 00000000 00:00 0 7fde58021000-7fde5c000000 ---p 00000000 00:00 0 7fde5c000000-7fde5c027000 rw-p 00000000 00:00 0 7fde5c027000-7fde60000000 ---p 00000000 00:00 0 7fde60000000-7fde60001000 rw-s 00000000 00:06 515 /dev/nvidia0 7fde60001000-7fde60002000 rw-s 00000000 00:06 515 /dev/nvidia0 7fde60002000-7fde60003000 rw-s 00000000 00:06 515 /dev/nvidia0 7fde60003000-7fde60004000 rw-s 00000000 00:06 515 /dev/nvidia0 7fde60004000-7fde60005000 rw-s 00000000 00:06 515 /dev/nvidia0 7fde60005000-7fde60006000 rw-s 00000000 00:06 515 /dev/nvidia0 7fde60006000-7fde60007000 rw-s 00000000 00:06 515 /dev/nvidia0 7fde60007000-7fde60008000 rw-s 00000000 00:06 515 /dev/nvidia0 7fde60008000-7fde60009000 rw-s 00000000 00:06 515 /dev/nvidia0 7fde60009000-7fde6000a000 rw-s 00000000 00:06 515 /dev/nvidia0 7fde6000a000-7fde6000b000 rw-s 00000000 00:06 515 /dev/nvidia0 7fde6000b000-7fde6000c000 rw-s 00000000 00:06 515 /dev/nvidia0 7fde6000c000-7fde6000d000 rw-s 00000000 00:06 515 /dev/nvidia0 7fde6000d000-7fde6000e000 rw-s 00000000 00:06 515 /dev/nvidia0 7fde6000e000-7fde6000f000 rw-s 00000000 00:06 515 /dev/nvidia0 7fde6000f000-7fde60010000 rw-s 00000000 00:06 515 /dev/nvidia0 7fde60010000-7fde70000000 ---p 00000000 00:00 0 7fde707fd000-7fde707fe000 ---p 00000000 00:00 0 7fde707fe000-7fde70ffe000 rw-p 00000000 00:00 0 7fde70ffe000-7fde70fff000 ---p 00000000 00:00 0 7fde70fff000-7fde717ff000 rw-p 00000000 00:00 0 7fde717ff000-7fde71800000 ---p 00000000 00:00 0 7fde71800000-7fde72000000 rw-p 00000000 00:00 0 7fde72000000-7fde74200000 ---p 00000000 00:00 0 7fde74200000-7fde74400000 rw-s 00000000 00:05 14087520 /dev/zero (deleted) 7fde74400000-7fde74600000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fde74600000-7fde74800000 rw-s 00000000 00:05 14087521 /dev/zero (deleted) 7fde74800000-7fde74a00000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fde74a00000-7fde74c00000 ---p 00000000 00:00 0 7fde74c00000-7fde74ed6000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fde74ed6000-7fde7c000000 ---p 00000000 00:00 0 7fde7c000000-7fde7c1b9000 rw-p 00000000 00:00 0 7fde7c1b9000-7fde80000000 ---p 00000000 00:00 0 7fde80000000-7fde80021000 rw-p 00000000 00:00 0 7fde80021000-7fde84000000 ---p 00000000 00:00 0 7fde84000000-7fde84227000 rw-p 00000000 00:00 0 7fde84227000-7fde88000000 ---p 00000000 00:00 0 7fde88000000-7fde881b9000 rw-p 00000000 00:00 0 7fde881b9000-7fde8c000000 ---p 00000000 00:00 0 7fde8c000000-7fde8c1b9000 rw-p 00000000 00:00 0 7fde8c1b9000-7fde90000000 ---p 00000000 00:00 0 7fde90000000-7fde901b9000 rw-p 00000000 00:00 0 7fde901b9000-7fde94000000 ---p 00000000 00:00 0 7fde94000000-7fde94227000 rw-p 00000000 00:00 0 7fde94227000-7fde98000000 ---p 00000000 00:00 0 7fde98000000-7fde98295000 rw-p 00000000 00:00 0 7fde98295000-7fde9c000000 ---p 00000000 00:00 0 7fde9c000000-7fde9c227000 rw-p 00000000 00:00 0 7fde9c227000-7fdea0000000 ---p 00000000 00:00 0 7fdea0000000-7fdea0371000 rw-p 00000000 00:00 0 7fdea0371000-7fdea4000000 ---p 00000000 00:00 0 7fdea4000000-7fdea41b9000 rw-p 00000000 00:00 0 7fdea41b9000-7fdea8000000 ---p 00000000 00:00 0 7fdea87f9000-7fdea87fa000 ---p 00000000 00:00 0 7fdea87fa000-7fdea8ffa000 rw-p 00000000 00:00 0 7fdea8ffa000-7fdea8ffb000 ---p 00000000 00:00 0 7fdea8ffb000-7fdea97fb000 rw-p 00000000 00:00 0 7fdea97fb000-7fdea97fc000 ---p 00000000 00:00 0 7fdea97fc000-7fdea9ffc000 rw-p 00000000 00:00 0 7fdea9ffc000-7fdea9ffd000 ---p 00000000 00:00 0 7fdea9ffd000-7fdeaa7fd000 rw-p 00000000 00:00 0 7fdeaa7fd000-7fdeaa7fe000 ---p 00000000 00:00 0 7fdeaa7fe000-7fdeaaffe000 rw-p 00000000 00:00 0 7fdeaaffe000-7fdeaafff000 ---p 00000000 00:00 0 7fdeaafff000-7fdeab7ff000 rw-p 00000000 00:00 0 7fdeab7ff000-7fdeab800000 ---p 00000000 00:00 0 7fdeab800000-7fdeac000000 rw-p 00000000 00:00 0 7fdeac000000-7fdeac295000 rw-p 00000000 00:00 0 7fdeac295000-7fdeb0000000 ---p 00000000 00:00 0 7fdeb0000000-7fdeb01b9000 rw-p 00000000 00:00 0 7fdeb01b9000-7fdeb4000000 ---p 00000000 00:00 0 7fdeb4000000-7fdeb4303000 rw-p 00000000 00:00 0 7fdeb4303000-7fdeb8000000 ---p 00000000 00:00 0 7fdeb8000000-7fdeb8295000 rw-p 00000000 00:00 0 7fdeb8295000-7fdebc000000 ---p 00000000 00:00 0 7fdebc000000-7fdebc227000 rw-p 00000000 00:00 0 7fdebc227000-7fdec0000000 ---p 00000000 00:00 0 7fdec0000000-7fdec01b9000 rw-p 00000000 00:00 0 7fdec01b9000-7fdec4000000 ---p 00000000 00:00 0 7fdec4000000-7fdec4303000 rw-p 00000000 00:00 0 7fdec4303000-7fdec8000000 ---p 00000000 00:00 0 7fdec8000000-7fdec81b9000 rw-p 00000000 00:00 0 7fdec81b9000-7fdecc000000 ---p 00000000 00:00 0 7fdecc000000-7fdecc227000 rw-p 00000000 00:00 0 7fdecc227000-7fded0000000 ---p 00000000 00:00 0 7fded0000000-7fded0227000 rw-p 00000000 00:00 0 7fded0227000-7fded4000000 ---p 00000000 00:00 0 7fded4000000-7fded4227000 rw-p 00000000 00:00 0 7fded4227000-7fded8000000 ---p 00000000 00:00 0 7fded8000000-7fded8295000 rw-p 00000000 00:00 0 7fded8295000-7fdedc000000 ---p 00000000 00:00 0 7fdedc000000-7fdedc295000 rw-p 00000000 00:00 0 7fdedc295000-7fdee0000000 ---p 00000000 00:00 0 7fdee0000000-7fdee0295000 rw-p 00000000 00:00 0 7fdee0295000-7fdee4000000 ---p 00000000 00:00 0 7fdee4000000-7fdee41b9000 rw-p 00000000 00:00 0 7fdee41b9000-7fdee8000000 ---p 00000000 00:00 0 7fdee8000000-7fdee81b9000 rw-p 00000000 00:00 0 7fdee81b9000-7fdeec000000 ---p 00000000 00:00 0 7fdeec000000-7fdeec227000 rw-p 00000000 00:00 0 7fdeec227000-7fdef0000000 ---p 00000000 00:00 0 7fdef0000000-7fdef01b9000 rw-p 00000000 00:00 0 7fdef01b9000-7fdef4000000 ---p 00000000 00:00 0 7fdef4000000-7fdef4227000 rw-p 00000000 00:00 0 7fdef4227000-7fdef8000000 ---p 00000000 00:00 0 7fdef8000000-7fdef8227000 rw-p 00000000 00:00 0 7fdef8227000-7fdefc000000 ---p 00000000 00:00 0 7fdefc000000-7fdefc295000 rw-p 00000000 00:00 0 7fdefc295000-7fdf00000000 ---p 00000000 00:00 0 7fdf00000000-7fdf00303000 rw-p 00000000 00:00 0 7fdf00303000-7fdf04000000 ---p 00000000 00:00 0 7fdf04000000-7fdf04227000 rw-p 00000000 00:00 0 7fdf04227000-7fdf08000000 ---p 00000000 00:00 0 7fdf087f9000-7fdf087fa000 ---p 00000000 00:00 0 7fdf087fa000-7fdf08ffa000 rw-p 00000000 00:00 0 7fdf08ffa000-7fdf08ffb000 ---p 00000000 00:00 0 7fdf08ffb000-7fdf097fb000 rw-p 00000000 00:00 0 7fdf097fb000-7fdf097fc000 ---p 00000000 00:00 0 7fdf097fc000-7fdf09ffc000 rw-p 00000000 00:00 0 7fdf09ffc000-7fdf09ffd000 ---p 00000000 00:00 0 7fdf09ffd000-7fdf0a7fd000 rw-p 00000000 00:00 0 7fdf0a7fd000-7fdf0a7fe000 ---p 00000000 00:00 0 7fdf0a7fe000-7fdf0affe000 rw-p 00000000 00:00 0 7fdf0affe000-7fdf0afff000 ---p 00000000 00:00 0 7fdf0afff000-7fdf0b7ff000 rw-p 00000000 00:00 0 7fdf0b7ff000-7fdf0b800000 ---p 00000000 00:00 0 7fdf0b800000-7fdf0c000000 rw-p 00000000 00:00 0 7fdf0c000000-7fdf0c227000 rw-p 00000000 00:00 0 7fdf0c227000-7fdf10000000 ---p 00000000 00:00 0 7fdf107f9000-7fdf107fa000 ---p 00000000 00:00 0 7fdf107fa000-7fdf10ffa000 rw-p 00000000 00:00 0 7fdf10ffa000-7fdf10ffb000 ---p 00000000 00:00 0 7fdf10ffb000-7fdf117fb000 rw-p 00000000 00:00 0 7fdf117fb000-7fdf117fc000 ---p 00000000 00:00 0 7fdf117fc000-7fdf11ffc000 rw-p 00000000 00:00 0 7fdf11ffc000-7fdf11ffd000 ---p 00000000 00:00 0 7fdf11ffd000-7fdf127fd000 rw-p 00000000 00:00 0 7fdf127fd000-7fdf127fe000 ---p 00000000 00:00 0 7fdf127fe000-7fdf12ffe000 rw-p 00000000 00:00 0 7fdf12ffe000-7fdf12fff000 ---p 00000000 00:00 0 7fdf12fff000-7fdf137ff000 rw-p 00000000 00:00 0 7fdf137ff000-7fdf13800000 ---p 00000000 00:00 0 7fdf13800000-7fdf14000000 rw-p 00000000 00:00 0 7fdf14000000-7fdf14295000 rw-p 00000000 00:00 0 7fdf14295000-7fdf18000000 ---p 00000000 00:00 0 7fdf18000000-7fdf18303000 rw-p 00000000 00:00 0 7fdf18303000-7fdf1c000000 ---p 00000000 00:00 0 7fdf1c000000-7fdf1c303000 rw-p 00000000 00:00 0 7fdf1c303000-7fdf20000000 ---p 00000000 00:00 0 7fdf207f9000-7fdf207fa000 ---p 00000000 00:00 0 7fdf207fa000-7fdf20ffa000 rw-p 00000000 00:00 0 7fdf20ffa000-7fdf20ffb000 ---p 00000000 00:00 0 7fdf20ffb000-7fdf217fb000 rw-p 00000000 00:00 0 7fdf217fb000-7fdf217fc000 ---p 00000000 00:00 0 7fdf217fc000-7fdf21ffc000 rw-p 00000000 00:00 0 7fdf21ffc000-7fdf21ffd000 ---p 00000000 00:00 0 7fdf21ffd000-7fdf227fd000 rw-p 00000000 00:00 0 7fdf227fd000-7fdf227fe000 ---p 00000000 00:00 0 7fdf227fe000-7fdf22ffe000 rw-p 00000000 00:00 0 7fdf22ffe000-7fdf22fff000 ---p 00000000 00:00 0 7fdf22fff000-7fdf237ff000 rw-p 00000000 00:00 0 7fdf237ff000-7fdf23800000 ---p 00000000 00:00 0 7fdf23800000-7fdf24000000 rw-p 00000000 00:00 0 7fdf24000000-7fdf24227000 rw-p 00000000 00:00 0 7fdf24227000-7fdf28000000 ---p 00000000 00:00 0 7fdf2872c000-7fdf2872d000 ---p 00000000 00:00 0 7fdf2872d000-7fdf28f2d000 rw-p 00000000 00:00 0 7fdf28f2d000-7fdf28f2e000 ---p 00000000 00:00 0 7fdf28f2e000-7fdf2972e000 rw-p 00000000 00:00 0 7fdf2972e000-7fdf2972f000 ---p 00000000 00:00 0 7fdf2972f000-7fdf29f2f000 rw-p 00000000 00:00 0 7fdf29f2f000-7fdf29f30000 ---p 00000000 00:00 0 7fdf29f30000-7fdf2a730000 rw-p 00000000 00:00 0 7fdf2a730000-7fdf2a731000 ---p 00000000 00:00 0 7fdf2a731000-7fdf2af31000 rw-p 00000000 00:00 0 7fdf2af31000-7fdf2af32000 ---p 00000000 00:00 0 7fdf2af32000-7fdf2b732000 rw-p 00000000 00:00 0 7fdf2b7ff000-7fdf2b800000 ---p 00000000 00:00 0 7fdf2b800000-7fdf2c000000 rw-p 00000000 00:00 0 7fdf2c000000-7fdf2c227000 rw-p 00000000 00:00 0 7fdf2c227000-7fdf30000000 ---p 00000000 00:00 0 7fdf3072c000-7fdf3072d000 ---p 00000000 00:00 0 7fdf3072d000-7fdf30f2d000 rw-p 00000000 00:00 0 7fdf30f2d000-7fdf30f2e000 ---p 00000000 00:00 0 7fdf30f2e000-7fdf3172e000 rw-p 00000000 00:00 0 7fdf3172e000-7fdf3172f000 ---p 00000000 00:00 0 7fdf3172f000-7fdf31f2f000 rw-p 00000000 00:00 0 7fdf31f2f000-7fdf31f30000 ---p 00000000 00:00 0 7fdf31f30000-7fdf32730000 rw-p 00000000 00:00 0 7fdf32730000-7fdf32731000 ---p 00000000 00:00 0 7fdf32731000-7fdf32f31000 rw-p 00000000 00:00 0 7fdf32f31000-7fdf32f32000 ---p 00000000 00:00 0 7fdf32f32000-7fdf33732000 rw-p 00000000 00:00 0 7fdf33732000-7fdf33733000 ---p 00000000 00:00 0 7fdf33733000-7fdf33f33000 rw-p 00000000 00:00 0 7fdf33f33000-7fdf33f34000 ---p 00000000 00:00 0 7fdf33f34000-7fdf34734000 rw-p 00000000 00:00 0 7fdf34734000-7fdf34735000 ---p 00000000 00:00 0 7fdf34735000-7fdf34f35000 rw-p 00000000 00:00 0 7fdf34f35000-7fdf34f36000 ---p 00000000 00:00 0 7fdf34f36000-7fdf35736000 rw-p 00000000 00:00 0 7fdf35a18000-7fdf35b8a000 r--p 00000000 08:11 16384476 /usr/lib/locale/C.UTF-8/LC_COLLATE 7fdf35b8a000-7fdf35e62000 r--p 00000000 08:11 16254002 /usr/lib/locale/locale-archive 7fdf35e62000-7fdf35e63000 ---p 00000000 00:00 0 7fdf35e63000-7fdf36663000 rw-p 00000000 00:00 0 7fdf36663000-7fdf36664000 ---p 00000000 00:00 0 7fdf36664000-7fdf36e64000 rw-p 00000000 00:00 0 7fdf36e64000-7fdf36e65000 ---p 00000000 00:00 0 7fdf36e65000-7fdf37665000 rw-p 00000000 00:00 0 7fdf37665000-7fdf37666000 ---p 00000000 00:00 0 7fdf37666000-7fdf37e66000 rw-p 00000000 00:00 0 7fdf37e66000-7fdf37e67000 ---p 00000000 00:00 0 7fdf37e67000-7fdf38667000 rw-p 00000000 00:00 0 7fdf38667000-7fdf38668000 ---p 00000000 00:00 0 7fdf38668000-7fdf38e68000 rw-p 00000000 00:00 0 7fdf38e68000-7fdf38e69000 ---p 00000000 00:00 0 7fdf38e69000-7fdf39669000 rw-p 00000000 00:00 0 7fdf39669000-7fdf3966a000 ---p 00000000 00:00 0 7fdf3966a000-7fdf3ac4e000 rw-p 00000000 00:00 0 7fdf3ac4e000-7fdf3ac59000 r-xp 00000000 08:11 14972856 /lib/x86_64-linux-gnu/libnss_files-2.23.so 7fdf3ac59000-7fdf3ae58000 ---p 0000b000 08:11 14972856 /lib/x86_64-linux-gnu/libnss_files-2.23.so 7fdf3ae58000-7fdf3ae59000 r--p 0000a000 08:11 14972856 /lib/x86_64-linux-gnu/libnss_files-2.23.so 7fdf3ae59000-7fdf3ae5a000 rw-p 0000b000 08:11 14972856 /lib/x86_64-linux-gnu/libnss_files-2.23.so 7fdf3ae5a000-7fdf3ae60000 rw-p 00000000 00:00 0 7fdf3ae60000-7fdf3ae62000 r-xp 00000000 08:11 14972850 /lib/x86_64-linux-gnu/libutil-2.23.so 7fdf3ae62000-7fdf3b061000 ---p 00002000 08:11 14972850 /lib/x86_64-linux-gnu/libutil-2.23.so 7fdf3b061000-7fdf3b062000 r--p 00001000 08:11 14972850 /lib/x86_64-linux-gnu/libutil-2.23.so 7fdf3b062000-7fdf3b063000 rw-p 00002000 08:11 14972850 /lib/x86_64-linux-gnu/libutil-2.23.so 7fdf3b063000-7fdf3b06d000 r-xp 00000000 08:11 16255033 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 7fdf3b06d000-7fdf3b26c000 ---p 0000a000 08:11 16255033 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 7fdf3b26c000-7fdf3b26d000 r--p 00009000 08:11 16255033 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 7fdf3b26d000-7fdf3b26e000 rw-p 0000a000 08:11 16255033 /usr/lib/x86_64-linux-gnu/libnuma.so.1.0.0 7fdf3b26e000-7fdf3b273000 r-xp 00000000 08:11 16258328 /usr/lib/x86_64-linux-gnu/libIlmThread-2_2.so.12.0.0 7fdf3b273000-7fdf3b473000 ---p 00005000 08:11 16258328 /usr/lib/x86_64-linux-gnu/libIlmThread-2_2.so.12.0.0 7fdf3b473000-7fdf3b474000 r--p 00005000 08:11 16258328 /usr/lib/x86_64-linux-gnu/libIlmThread-2_2.so.12.0.0 7fdf3b474000-7fdf3b475000 rw-p 00006000 08:11 16258328 /usr/lib/x86_64-linux-gnu/libIlmThread-2_2.so.12.0.0 7fdf3b475000-7fdf3b490000 r-xp 00000000 08:11 16258325 /usr/lib/x86_64-linux-gnu/libIex-2_2.so.12.0.0 7fdf3b490000-7fdf3b68f000 ---p 0001b000 08:11 16258325 /usr/lib/x86_64-linux-gnu/libIex-2_2.so.12.0.0 7fdf3b68f000-7fdf3b692000 r--p 0001a000 08:11 16258325 /usr/lib/x86_64-linux-gnu/libIex-2_2.so.12.0.0 7fdf3b692000-7fdf3b693000 rw-p 0001d000 08:11 16258325 /usr/lib/x86_64-linux-gnu/libIex-2_2.so.12.0.0 7fdf3b693000-7fdf3b6d5000 r-xp 00000000 08:11 16258320 /usr/lib/x86_64-linux-gnu/libHalf.so.12.0.0 7fdf3b6d5000-7fdf3b8d4000 ---p 00042000 08:11 16258320 /usr/lib/x86_64-linux-gnu/libHalf.so.12.0.0 7fdf3b8d4000-7fdf3b8d5000 r--p 00041000 08:11 16258320 /usr/lib/x86_64-linux-gnu/libHalf.so.12.0.0 7fdf3b8d5000-7fdf3b8d6000 rw-p 00042000 08:11 16258320 /usr/lib/x86_64-linux-gnu/libHalf.so.12.0.0 7fdf3b8d6000-7fdf3b8e1000 r-xp 00000000 08:11 16254977 /usr/lib/x86_64-linux-gnu/libjbig.so.0 7fdf3b8e1000-7fdf3bae0000 ---p 0000b000 08:11 16254977 /usr/lib/x86_64-linux-gnu/libjbig.so.0 7fdf3bae0000-7fdf3bae1000 r--p 0000a000 08:11 16254977 /usr/lib/x86_64-linux-gnu/libjbig.so.0 7fdf3bae1000-7fdf3bae4000 rw-p 0000b000 08:11 16254977 /usr/lib/x86_64-linux-gnu/libjbig.so.0 7fdf3bae4000-7fdf3bb05000 r-xp 00000000 08:11 14946432 /lib/x86_64-linux-gnu/liblzma.so.5.0.0 7fdf3bb05000-7fdf3bd04000 ---p 00021000 08:11 14946432 /lib/x86_64-linux-gnu/liblzma.so.5.0.0 7fdf3bd04000-7fdf3bd05000 r--p 00020000 08:11 14946432 /lib/x86_64-linux-gnu/liblzma.so.5.0.0 7fdf3bd05000-7fdf3bd06000 rw-p 00021000 08:11 14946432 /lib/x86_64-linux-gnu/liblzma.so.5.0.0 7fdf3bd06000-7fdf3bdd8000 r-xp 00000000 08:11 16650780 /usr/local/lib/libopen-pal.so.13.0.3 7fdf3bdd8000-7fdf3bfd8000 ---p 000d2000 08:11 16650780 /usr/local/lib/libopen-pal.so.13.0.3 7fdf3bfd8000-7fdf3bfdc000 r--p 000d2000 08:11 16650780 /usr/local/lib/libopen-pal.so.13.0.3 7fdf3bfdc000-7fdf3bfe1000 rw-p 000d6000 08:11 16650780 /usr/local/lib/libopen-pal.so.13.0.3 7fdf3bfe1000-7fdf3bfe7000 rw-p 00000000 00:00 0 7fdf3bfe7000-7fdf3c05e000 r-xp 00000000 08:11 16650787 /usr/local/lib/libopen-rte.so.12.0.3 7fdf3c05e000-7fdf3c25e000 ---p 00077000 08:11 16650787 /usr/local/lib/libopen-rte.so.12.0.3 7fdf3c25e000-7fdf3c25f000 r--p 00077000 08:11 16650787 /usr/local/lib/libopen-rte.so.12.0.3 7fdf3c25f000-7fdf3c262000 rw-p 00078000 08:11 16650787 /usr/local/lib/libopen-rte.so.12.0.3 7fdf3c262000-7fdf3c264000 rw-p 00000000 00:00 0 7fdf3c264000-7fdf3c271000 r-xp 00000000 08:11 16261513 /usr/lib/libibverbs.so.1.0.0 7fdf3c271000-7fdf3c471000 ---p 0000d000 08:11 16261513 /usr/lib/libibverbs.so.1.0.0 7fdf3c471000-7fdf3c472000 r--p 0000d000 08:11 16261513 /usr/lib/libibverbs.so.1.0.0 7fdf3c472000-7fdf3c473000 rw-p 0000e000 08:11 16261513 /usr/lib/libibverbs.so.1.0.0 7fdf3c473000-7fdf3c49c000 r-xp 00000000 08:11 21294873 /home/tjb3/apps/cntk/build/release/lib/libCntk.PerformanceProfiler-2.5.1.so 7fdf3c49c000-7fdf3c69b000 ---p 00029000 08:11 21294873 /home/tjb3/apps/cntk/build/release/lib/libCntk.PerformanceProfiler-2.5.1.so 7fdf3c69b000-7fdf3c69c000 r--p 00028000 08:11 21294873 /home/tjb3/apps/cntk/build/release/lib/libCntk.PerformanceProfiler-2.5.1.so 7fdf3c69c000-7fdf3c69d000 rw-p 00029000 08:11 21294873 /home/tjb3/apps/cntk/build/release/lib/libCntk.PerformanceProfiler-2.5.1.so 7fdf3c69d000-7fdf45463000 r-xp 00000000 08:11 17567731 /usr/local/mklml/mklml_lnx_2018.0.3.20180406/lib/libmklml_intel.so 7fdf45463000-7fdf45663000 ---p 08dc6000 08:11 17567731 /usr/local/mklml/mklml_lnx_2018.0.3.20180406/lib/libmklml_intel.so 7fdf45663000-7fdf45685000 r--p 08dc6000 08:11 17567731 /usr/local/mklml/mklml_lnx_2018.0.3.20180406/lib/libmklml_intel.so 7fdf45685000-7fdf4574a000 rw-p 08de8000 08:11 17567731 /usr/local/mklml/mklml_lnx_2018.0.3.20180406/lib/libmklml_intel.so 7fdf4574a000-7fdf4576e000 rw-p 00000000 00:00 0 7fdf4576e000-7fdf569b3000 r-xp 00000000 08:11 16253523 /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudnn.so.7.2.1 7fdf569b3000-7fdf56bb3000 ---p 11245000 08:11 16253523 /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudnn.so.7.2.1 7fdf56bb3000-7fdf56c18000 rw-p 11245000 08:11 16253523 /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudnn.so.7.2.1 7fdf56c18000-7fdf56c75000 rw-p 00000000 00:00 0 7fdf56c75000-7fdf5a1b1000 r-xp 00000000 08:11 16300891 /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcusparse.so.9.0.176 7fdf5a1b1000-7fdf5a3b1000 ---p 0353c000 08:11 16300891 /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcusparse.so.9.0.176 7fdf5a3b1000-7fdf5a3cb000 rw-p 0353c000 08:11 16300891 /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcusparse.so.9.0.176 7fdf5a3cb000-7fdf5a3db000 rw-p 00000000 00:00 0 7fdf5a3db000-7fdf5c864000 r-xp 00000000 08:11 16300867 /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcurand.so.9.0.176 7fdf5c864000-7fdf5ca63000 ---p 02489000 08:11 16300867 /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcurand.so.9.0.176 7fdf5ca63000-7fdf5de35000 rw-p 02488000 08:11 16300867 /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcurand.so.9.0.176 7fdf5de35000-7fdf5e33f000 rw-p 00000000 00:00 0 7fdf5e33f000-7fdf61530000 r-xp 00000000 08:11 16300838 /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcublas.so.9.0.176 7fdf61530000-7fdf6172f000 ---p 031f1000 08:11 16300838 /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcublas.so.9.0.176 7fdf6172f000-7fdf61766000 rw-p 031f0000 08:11 16300838 /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcublas.so.9.0.176 7fdf61766000-7fdf61775000 rw-p 00000000 00:00 0 7fdf61775000-7fdf6198f000 r-xp 00000000 08:11 14942295 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 7fdf6198f000-7fdf61b8e000 ---p 0021a000 08:11 14942295 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 7fdf61b8e000-7fdf61baa000 r--p 00219000 08:11 14942295 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 7fdf61baa000-7fdf61bb6000 rw-p 00235000 08:11 14942295 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 7fdf61bb6000-7fdf61bb9000 rw-p 00000000 00:00 0 7fdf61bb9000-7fdf61c17000 r-xp 00000000 08:11 14942426 /lib/x86_64-linux-gnu/libssl.so.1.0.0 7fdf61c17000-7fdf61e17000 ---p 0005e000 08:11 14942426 /lib/x86_64-linux-gnu/libssl.so.1.0.0 7fdf61e17000-7fdf61e1b000 r--p 0005e000 08:11 14942426 /lib/x86_64-linux-gnu/libssl.so.1.0.0 7fdf61e1b000-7fdf61e22000 rw-p 00062000 08:11 14942426 /lib/x86_64-linux-gnu/libssl.so.1.0.0 7fdf61e22000-7fdf61e2e000 r-xp 00000000 08:11 16257197 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 7fdf61e2e000-7fdf6202d000 ---p 0000c000 08:11 16257197 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 7fdf6202d000-7fdf6202e000 r--p 0000b000 08:11 16257197 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 7fdf6202e000-7fdf6202f000 rw-p 0000c000 08:11 16257197 /usr/lib/x86_64-linux-gnu/libunwind.so.8.0.1 7fdf6202f000-7fdf6203d000 rw-p 00000000 00:00 0 7fdf6203d000-7fdf6205d000 r-xp 00000000 08:11 16256931 /usr/lib/x86_64-linux-gnu/libgflags.so.2.1.2 7fdf6205d000-7fdf6225c000 ---p 00020000 08:11 16256931 /usr/lib/x86_64-linux-gnu/libgflags.so.2.1.2 7fdf6225c000-7fdf6225d000 r--p 0001f000 08:11 16256931 /usr/lib/x86_64-linux-gnu/libgflags.so.2.1.2 7fdf6225d000-7fdf6225e000 rw-p 00020000 08:11 16256931 /usr/lib/x86_64-linux-gnu/libgflags.so.2.1.2 7fdf6225e000-7fdf62428000 r-xp 00000000 08:11 16260479 /usr/lib/x86_64-linux-gnu/libIlmImf-2_2.so.22.0.0 7fdf62428000-7fdf62627000 ---p 001ca000 08:11 16260479 /usr/lib/x86_64-linux-gnu/libIlmImf-2_2.so.22.0.0 7fdf62627000-7fdf6262a000 r--p 001c9000 08:11 16260479 /usr/lib/x86_64-linux-gnu/libIlmImf-2_2.so.22.0.0 7fdf6262a000-7fdf6272b000 rw-p 001cc000 08:11 16260479 /usr/lib/x86_64-linux-gnu/libIlmImf-2_2.so.22.0.0 7fdf6272b000-7fdf6272c000 rw-p 00000000 00:00 0 7fdf6272c000-7fdf62776000 r-xp 00000000 08:11 16287672 /usr/lib/x86_64-linux-gnu/libjasper.so.1.0.0 7fdf62776000-7fdf62975000 ---p 0004a000 08:11 16287672 /usr/lib/x86_64-linux-gnu/libjasper.so.1.0.0 7fdf62975000-7fdf62976000 r--p 00049000 08:11 16287672 /usr/lib/x86_64-linux-gnu/libjasper.so.1.0.0 7fdf62976000-7fdf6297a000 rw-p 0004a000 08:11 16287672 /usr/lib/x86_64-linux-gnu/libjasper.so.1.0.0 7fdf6297a000-7fdf62981000 rw-p 00000000 00:00 0 7fdf62981000-7fdf629f2000 r-xp 00000000 08:11 16255181 /usr/lib/x86_64-linux-gnu/libtiff.so.5.2.4 7fdf629f2000-7fdf62bf1000 ---p 00071000 08:11 16255181 /usr/lib/x86_64-linux-gnu/libtiff.so.5.2.4 7fdf62bf1000-7fdf62bf2000 r--p 00070000 08:11 16255181 /usr/lib/x86_64-linux-gnu/libtiff.so.5.2.4 7fdf62bf2000-7fdf62bf5000 rw-p 00071000 08:11 16255181 /usr/lib/x86_64-linux-gnu/libtiff.so.5.2.4 7fdf62bf5000-7fdf62c19000 r-xp 00000000 08:11 14942299 /lib/x86_64-linux-gnu/libpng12.so.0.54.0 7fdf62c19000-7fdf62e18000 ---p 00024000 08:11 14942299 /lib/x86_64-linux-gnu/libpng12.so.0.54.0 7fdf62e18000-7fdf62e19000 r--p 00023000 08:11 14942299 /lib/x86_64-linux-gnu/libpng12.so.0.54.0 7fdf62e19000-7fdf62e1a000 rw-p 00024000 08:11 14942299 /lib/x86_64-linux-gnu/libpng12.so.0.54.0 7fdf62e1a000-7fdf62e71000 r-xp 00000000 08:11 16254218 /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2 7fdf62e71000-7fdf63071000 ---p 00057000 08:11 16254218 /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2 7fdf63071000-7fdf63072000 r--p 00057000 08:11 16254218 /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2 7fdf63072000-7fdf63073000 rw-p 00058000 08:11 16254218 /usr/lib/x86_64-linux-gnu/libjpeg.so.8.0.2 7fdf63073000-7fdf6308c000 r-xp 00000000 08:11 14942258 /lib/x86_64-linux-gnu/libz.so.1.2.8 7fdf6308c000-7fdf6328b000 ---p 00019000 08:11 14942258 /lib/x86_64-linux-gnu/libz.so.1.2.8 7fdf6328b000-7fdf6328c000 r--p 00018000 08:11 14942258 /lib/x86_64-linux-gnu/libz.so.1.2.8 7fdf6328c000-7fdf6328d000 rw-p 00019000 08:11 14942258 /lib/x86_64-linux-gnu/libz.so.1.2.8 7fdf6328d000-7fdf6333f000 r-xp 00000000 08:11 17042307 /usr/lib/openmpi/lib/libmpi.so.12.0.2 7fdf6333f000-7fdf6353f000 ---p 000b2000 08:11 17042307 /usr/lib/openmpi/lib/libmpi.so.12.0.2 7fdf6353f000-7fdf63540000 r--p 000b2000 08:11 17042307 /usr/lib/openmpi/lib/libmpi.so.12.0.2 7fdf63540000-7fdf63550000 rw-p 000b3000 08:11 17042307 /usr/lib/openmpi/lib/libmpi.so.12.0.2 7fdf63550000-7fdf63563000 rw-p 00000000 00:00 0 7fdf63563000-7fdf6357b000 r-xp 00000000 08:11 17041447 /usr/lib/openmpi/lib/libmpi_cxx.so.1.1.3 7fdf6357b000-7fdf6377b000 ---p 00018000 08:11 17041447 /usr/lib/openmpi/lib/libmpi_cxx.so.1.1.3 7fdf6377b000-7fdf6377d000 r--p 00018000 08:11 17041447 /usr/lib/openmpi/lib/libmpi_cxx.so.1.1.3 7fdf6377d000-7fdf6377e000 rw-p 0001a000 08:11 17041447 /usr/lib/openmpi/lib/libmpi_cxx.so.1.1.3 7fdf6377e000-7fdf6abad000 r-xp 00000000 08:11 21295197 /home/tjb3/apps/cntk/build/release/lib/libCntk.Math-2.5.1.so 7fdf6abad000-7fdf6adad000 ---p 0742f000 08:11 21295197 /home/tjb3/apps/cntk/build/release/lib/libCntk.Math-2.5.1.so 7fdf6adad000-7fdf6adb7000 r--p 0742f000 08:11 21295197 /home/tjb3/apps/cntk/build/release/lib/libCntk.Math-2.5.1.so 7fdf6adb7000-7fdf6adbf000 rw-p 07439000 08:11 21295197 /home/tjb3/apps/cntk/build/release/lib/libCntk.Math-2.5.1.so 7fdf6adbf000-7fdf6adfa000 rw-p 00000000 00:00 0 7fdf6adfa000-7fdf6af64000 r-xp 00000000 08:11 17567730 /usr/local/mklml/mklml_lnx_2018.0.3.20180406/lib/libiomp5.so 7fdf6af64000-7fdf6b164000 ---p 0016a000 08:11 17567730 /usr/local/mklml/mklml_lnx_2018.0.3.20180406/lib/libiomp5.so 7fdf6b164000-7fdf6b16f000 rw-p 0016a000 08:11 17567730 /usr/local/mklml/mklml_lnx_2018.0.3.20180406/lib/libiomp5.so 7fdf6b16f000-7fdf6b19e000 rw-p 00000000 00:00 0 7fdf6b19e000-7fdf6b2de000 r-xp 00000000 08:11 16287708 /usr/lib/nvidia-396/libnvidia-ml.so.396.45 7fdf6b2de000-7fdf6b4de000 ---p 00140000 08:11 16287708 /usr/lib/nvidia-396/libnvidia-ml.so.396.45 7fdf6b4de000-7fdf6b4f4000 rw-p 00140000 08:11 16287708 /usr/lib/nvidia-396/libnvidia-ml.so.396.45 7fdf6b4f4000-7fdf6b78d000 rw-p 00000000 00:00 0 7fdf6b78d000-7fdf6b7f6000 r-xp 00000000 08:11 16300575 /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudart.so.9.0.176 7fdf6b7f6000-7fdf6b9f5000 ---p 00069000 08:11 16300575 /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudart.so.9.0.176 7fdf6b9f5000-7fdf6b9f9000 rw-p 00068000 08:11 16300575 /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudart.so.9.0.176 7fdf6b9f9000-7fdf6b9fa000 rw-p 00000000 00:00 0 7fdf6b9fa000-7fdf6ba37000 r-xp 00000000 08:11 16287701 /usr/lib/nvidia-396/libnvidia-fatbinaryloader.so.396.45 7fdf6ba37000-7fdf6bc36000 ---p 0003d000 08:11 16287701 /usr/lib/nvidia-396/libnvidia-fatbinaryloader.so.396.45 7fdf6bc36000-7fdf6bc41000 rw-p 0003c000 08:11 16287701 /usr/lib/nvidia-396/libnvidia-fatbinaryloader.so.396.45 7fdf6bc41000-7fdf6bc46000 rw-p 00000000 00:00 0 7fdf6bc46000-7fdf6be06000 r-xp 00000000 08:11 14972843 /lib/x86_64-linux-gnu/libc-2.23.so 7fdf6be06000-7fdf6c006000 ---p 001c0000 08:11 14972843 /lib/x86_64-linux-gnu/libc-2.23.so 7fdf6c006000-7fdf6c00a000 r--p 001c0000 08:11 14972843 /lib/x86_64-linux-gnu/libc-2.23.so 7fdf6c00a000-7fdf6c00c000 rw-p 001c4000 08:11 14972843 /lib/x86_64-linux-gnu/libc-2.23.so 7fdf6c00c000-7fdf6c010000 rw-p 00000000 00:00 0 7fdf6c010000-7fdf6c026000 r-xp 00000000 08:11 14942285 /lib/x86_64-linux-gnu/libgcc_s.so.1 7fdf6c026000-7fdf6c225000 ---p 00016000 08:11 14942285 /lib/x86_64-linux-gnu/libgcc_s.so.1 7fdf6c225000-7fdf6c226000 rw-p 00015000 08:11 14942285 /lib/x86_64-linux-gnu/libgcc_s.so.1 7fdf6c226000-7fdf6c398000 r-xp 00000000 08:11 16254956 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21 7fdf6c398000-7fdf6c598000 ---p 00172000 08:11 16254956 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21 7fdf6c598000-7fdf6c5a2000 r--p 00172000 08:11 16254956 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21 7fdf6c5a2000-7fdf6c5a4000 rw-p 0017c000 08:11 16254956 /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21 7fdf6c5a4000-7fdf6c5a8000 rw-p 00000000 00:00 0 7fdf6c5a8000-7fdf6c6b0000 r-xp 00000000 08:11 14942246 /lib/x86_64-linux-gnu/libm-2.23.so 7fdf6c6b0000-7fdf6c8af000 ---p 00108000 08:11 14942246 /lib/x86_64-linux-gnu/libm-2.23.so 7fdf6c8af000-7fdf6c8b0000 r--p 00107000 08:11 14942246 /lib/x86_64-linux-gnu/libm-2.23.so 7fdf6c8b0000-7fdf6c8b1000 rw-p 00108000 08:11 14942246 /lib/x86_64-linux-gnu/libm-2.23.so 7fdf6c8b1000-7fdf6c8c9000 r-xp 00000000 08:11 14972842 /lib/x86_64-linux-gnu/libpthread-2.23.so 7fdf6c8c9000-7fdf6cac8000 ---p 00018000 08:11 14972842 /lib/x86_64-linux-gnu/libpthread-2.23.so 7fdf6cac8000-7fdf6cac9000 r--p 00017000 08:11 14972842 /lib/x86_64-linux-gnu/libpthread-2.23.so 7fdf6cac9000-7fdf6caca000 rw-p 00018000 08:11 14972842 /lib/x86_64-linux-gnu/libpthread-2.23.so 7fdf6caca000-7fdf6cace000 rw-p 00000000 00:00 0 7fdf6cace000-7fdf6cb27000 r-xp 00000000 08:11 23594806 /home/tjb3/work/MITS/src/HTGS-MITS/lib/mariadb/libmariadb.so.3 7fdf6cb27000-7fdf6cd27000 ---p 00059000 08:11 23594806 /home/tjb3/work/MITS/src/HTGS-MITS/lib/mariadb/libmariadb.so.3 7fdf6cd27000-7fdf6cd2d000 r--p 00059000 08:11 23594806 /home/tjb3/work/MITS/src/HTGS-MITS/lib/mariadb/libmariadb.so.3 7fdf6cd2d000-7fdf6cd30000 rw-p 0005f000 08:11 23594806 /home/tjb3/work/MITS/src/HTGS-MITS/lib/mariadb/libmariadb.so.3 7fdf6cd30000-7fdf6cd31000 rw-p 00000000 00:00 0 7fdf6cd31000-7fdf6cd4f000 r-xp 00000000 08:11 16257200 /usr/lib/x86_64-linux-gnu/libglog.so.0.0.0 7fdf6cd4f000-7fdf6cf4e000 ---p 0001e000 08:11 16257200 /usr/lib/x86_64-linux-gnu/libglog.so.0.0.0 7fdf6cf4e000-7fdf6cf4f000 r--p 0001d000 08:11 16257200 /usr/lib/x86_64-linux-gnu/libglog.so.0.0.0 7fdf6cf4f000-7fdf6cf50000 rw-p 0001e000 08:11 16257200 /usr/lib/x86_64-linux-gnu/libglog.so.0.0.0 7fdf6cf50000-7fdf6cf60000 rw-p 00000000 00:00 0 7fdf6cf60000-7fdf6d039000 r-xp 00000000 08:11 16680877 /usr/local/lib/libopencv_imgcodecs.so.3.4.2 7fdf6d039000-7fdf6d238000 ---p 000d9000 08:11 16680877 /usr/local/lib/libopencv_imgcodecs.so.3.4.2 7fdf6d238000-7fdf6d23b000 r--p 000d8000 08:11 16680877 /usr/local/lib/libopencv_imgcodecs.so.3.4.2 7fdf6d23b000-7fdf6d23c000 rw-p 000db000 08:11 16680877 /usr/local/lib/libopencv_imgcodecs.so.3.4.2 7fdf6d23c000-7fdf6d23e000 rw-p 00000000 00:00 0 7fdf6d23e000-7fdf6d62c000 r-xp 00000000 08:11 16675531 /usr/local/lib/libopencv_imgproc.so.3.4.2 7fdf6d62c000-7fdf6d82c000 ---p 003ee000 08:11 16675531 /usr/local/lib/libopencv_imgproc.so.3.4.2 7fdf6d82c000-7fdf6d835000 r--p 003ee000 08:11 16675531 /usr/local/lib/libopencv_imgproc.so.3.4.2 7fdf6d835000-7fdf6d83e000 rw-p 003f7000 08:11 16675531 /usr/local/lib/libopencv_imgproc.so.3.4.2 7fdf6d83e000-7fdf6d8dd000 rw-p 00000000 00:00 0 7fdf6d8dd000-7fdf6dc27000 r-xp 00000000 08:11 16668812 /usr/local/lib/libopencv_core.so.3.4.2 7fdf6dc27000-7fdf6de26000 ---p 0034a000 08:11 16668812 /usr/local/lib/libopencv_core.so.3.4.2 7fdf6de26000-7fdf6de2e000 r--p 00349000 08:11 16668812 /usr/local/lib/libopencv_core.so.3.4.2 7fdf6de2e000-7fdf6de33000 rw-p 00351000 08:11 16668812 /usr/local/lib/libopencv_core.so.3.4.2 7fdf6de33000-7fdf6de38000 rw-p 00000000 00:00 0 7fdf6de38000-7fdf6df32000 r-xp 00000000 08:11 21160658 /home/tjb3/apps/libCZI/build/Src/libCZI/liblibCZI.so 7fdf6df32000-7fdf6e131000 ---p 000fa000 08:11 21160658 /home/tjb3/apps/libCZI/build/Src/libCZI/liblibCZI.so 7fdf6e131000-7fdf6e135000 r--p 000f9000 08:11 21160658 /home/tjb3/apps/libCZI/build/Src/libCZI/liblibCZI.so 7fdf6e135000-7fdf6e138000 rw-p 000fd000 08:11 21160658 /home/tjb3/apps/libCZI/build/Src/libCZI/liblibCZI.so 7fdf6e138000-7fdf6f428000 r-xp 00000000 08:11 21295437 /home/tjb3/apps/cntk/build/release/lib/libCntk.Core-2.5.1.so 7fdf6f428000-7fdf6f628000 ---p 012f0000 08:11 21295437 /home/tjb3/apps/cntk/build/release/lib/libCntk.Core-2.5.1.so 7fdf6f628000-7fdf6f6a4000 r--p 012f0000 08:11 21295437 /home/tjb3/apps/cntk/build/release/lib/libCntk.Core-2.5.1.so 7fdf6f6a4000-7fdf6f6c0000 rw-p 0136c000 08:11 21295437 /home/tjb3/apps/cntk/build/release/lib/libCntk.Core-2.5.1.so 7fdf6f6c0000-7fdf6f6ca000 rw-p 00000000 00:00 0 7fdf6f6ca000-7fdf702d1000 r-xp 00000000 08:11 16254839 /usr/lib/x86_64-linux-gnu/libcuda.so.396.45 7fdf702d1000-7fdf704d0000 ---p 00c07000 08:11 16254839 /usr/lib/x86_64-linux-gnu/libcuda.so.396.45 7fdf704d0000-7fdf70636000 rw-p 00c06000 08:11 16254839 /usr/lib/x86_64-linux-gnu/libcuda.so.396.45 7fdf70636000-7fdf70645000 rw-p 00000000 00:00 0 7fdf70645000-7fdf7064c000 r-xp 00000000 08:11 14972861 /lib/x86_64-linux-gnu/librt-2.23.so 7fdf7064c000-7fdf7084b000 ---p 00007000 08:11 14972861 /lib/x86_64-linux-gnu/librt-2.23.so 7fdf7084b000-7fdf7084c000 r--p 00006000 08:11 14972861 /lib/x86_64-linux-gnu/librt-2.23.so 7fdf7084c000-7fdf7084d000 rw-p 00007000 08:11 14972861 /lib/x86_64-linux-gnu/librt-2.23.so 7fdf7084d000-7fdf70850000 r-xp 00000000 08:11 14972845 /lib/x86_64-linux-gnu/libdl-2.23.so 7fdf70850000-7fdf70a4f000 ---p 00003000 08:11 14972845 /lib/x86_64-linux-gnu/libdl-2.23.so 7fdf70a4f000-7fdf70a50000 r--p 00002000 08:11 14972845 /lib/x86_64-linux-gnu/libdl-2.23.so 7fdf70a50000-7fdf70a51000 rw-p 00003000 08:11 14972845 /lib/x86_64-linux-gnu/libdl-2.23.so 7fdf70a51000-7fdf70a77000 r-xp 00000000 08:11 14972841 /lib/x86_64-linux-gnu/ld-2.23.so 7fdf70b3c000-7fdf70b9d000 rw-p 00000000 00:00 0 7fdf70bf9000-7fdf70c20000 r--p 00000000 08:11 16384479 /usr/lib/locale/C.UTF-8/LC_CTYPE 7fdf70c20000-7fdf70c3f000 rw-p 00000000 00:00 0 7fdf70c53000-7fdf70c54000 rw-p 00000000 00:00 0 7fdf70c54000-7fdf70c55000 r--p 00000000 08:11 16384467 /usr/lib/locale/C.UTF-8/LC_NUMERIC 7fdf70c55000-7fdf70c56000 r--p 00000000 08:11 16384481 /usr/lib/locale/C.UTF-8/LC_TIME 7fdf70c56000-7fdf70c57000 r--p 00000000 08:11 16384464 /usr/lib/locale/C.UTF-8/LC_MONETARY 7fdf70c57000-7fdf70c58000 r--p 00000000 08:11 16384480 /usr/lib/locale/C.UTF-8/LC_MESSAGES/SYS_LC_MESSAGES 7fdf70c58000-7fdf70c59000 r--p 00000000 08:11 16384477 /usr/lib/locale/C.UTF-8/LC_PAPER 7fdf70c59000-7fdf70c5a000 r--p 00000000 08:11 16384469 /usr/lib/locale/C.UTF-8/LC_NAME 7fdf70c5a000-7fdf70c5b000 r--p 00000000 08:11 16384466 /usr/lib/locale/C.UTF-8/LC_ADDRESS 7fdf70c5b000-7fdf70c5c000 r--p 00000000 08:11 16384465 /usr/lib/locale/C.UTF-8/LC_TELEPHONE 7fdf70c5c000-7fdf70c5d000 r--p 00000000 08:11 16384478 /usr/lib/locale/C.UTF-8/LC_MEASUREMENT 7fdf70c5d000-7fdf70c64000 r--s 00000000 08:11 16650923 /usr/lib/x86_64-linux-gnu/gconv/gconv-modules.cache 7fdf70c64000-7fdf70c65000 r--p 00000000 08:11 16384468 /usr/lib/locale/C.UTF-8/LC_IDENTIFICATION 7fdf70c65000-7fdf70c66000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fdf70c66000-7fdf70c67000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fdf70c67000-7fdf70c68000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fdf70c68000-7fdf70c69000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fdf70c69000-7fdf70c6a000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fdf70c6a000-7fdf70c6b000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fdf70c6b000-7fdf70c6c000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fdf70c6c000-7fdf70c6d000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fdf70c6d000-7fdf70c6e000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fdf70c6e000-7fdf70c6f000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fdf70c6f000-7fdf70c70000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fdf70c70000-7fdf70c71000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fdf70c71000-7fdf70c72000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fdf70c72000-7fdf70c73000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fdf70c73000-7fdf70c74000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fdf70c74000-7fdf70c75000 rw-s 00000000 00:06 514 /dev/nvidiactl 7fdf70c75000-7fdf70c76000 rw-p 00000000 00:00 0 7fdf70c76000-7fdf70c77000 r--p 00025000 08:11 14972841 /lib/x86_64-linux-gnu/ld-2.23.so 7fdf70c77000-7fdf70c78000 rw-p 00026000 08:11 14972841 /lib/x86_64-linux-gnu/ld-2.23.so 7fdf70c78000-7fdf70c79000 rw-p 00000000 00:00 0 7ffee31d6000-7ffee31f7000 rw-p 00000000 00:00 0 [stack] 7ffee31f7000-7ffee31fa000 r--p 00000000 00:00 0 [vvar] 7ffee31fa000-7ffee31fc000 r-xp 00000000 00:00 0 [vdso] ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall]

kit1980 commented 6 years ago

Is it possible for you to share the ONNX model from Caffe2?

tblattner commented 6 years ago

Attached is the onnx model that I am attempting to load. Thanks.

model.zip

kit1980 commented 6 years ago

The problematic node is Squeeze, I'll look how to fix this.

mmajurski commented 6 years ago

The Squeeze was added to the model because the output of the last convolution layer (well actually the average pool after the conv layer) has shape [-1, 1024, 1, 1].

Without the squeeze the onnx converted added a reshape layer to format the input for the fc layer which is expecting the input to be of shape [-1, 1024].

This reshape layer was causing import problems when loading the onnx file into tensorRT. So we tried to explicitly add the squeeze as it appeared to be on the supported layer list for tensorRT. However, having solved that issue we have run into other show stopping issues with tensorRT.

I am retraining now without the squeeze. I assume that Caffe2 will insert a reshape to convert [-1, 1024, 1, 1] to [-1, 1024]. We can generate a new onnx file without the squeeze as soon as it checkpoints.

Thanks, Michael (tblattner's colleague)

tblattner commented 6 years ago

Attached is the onnx model without squeeze. The Caffe2 to onnx converter added an implicit Reshape operator.

model-nosqeeze.zip

Here is the code we used to do the conversion:

import caffe2.python
import caffe2.python.onnx.frontend
from caffe2.proto import caffe2_pb2
from caffe2.python import workspace
import numpy as np
import onnx
import google.protobuf.text_format

workspace.ResetWorkspace()

INIT_NET = '/path/to//init_net-0.pb'
PREDICT_NET = '/path/to/predict_net-0-cpu.pbtxt'

tile_size = 128

# We need to provide type and shape of the model inputs,
# see above Note section for explanation
data_type = onnx.TensorProto.FLOAT
data_shape = (1, 1, tile_size, tile_size)
value_info = {
    'data': (data_type, data_shape)
}

init_net = caffe2_pb2.NetDef()
with open(INIT_NET, 'rb') as f:
    init_net.ParseFromString(f.read())

predict_net = caffe2_pb2.NetDef()
with open(PREDICT_NET, 'r') as f:
    # this is how you parse a protobuf from text format
    predict_net = google.protobuf.text_format.Parse(f.read(), predict_net)
    # predict_net.ParseFromString(f.read()) # this parse from string is actually for binary data in a string container

onnx_model = caffe2.python.onnx.frontend.caffe2_net_to_onnx_model(
    predict_net,
    init_net,
    value_info,
)

onnx.checker.check_model(onnx_model)

onnx.save(onnx_model, './inception-model.onnx')

Edit: Unfortunately, running the CNTK model loader on the nosqueeze version still produced a dump.

kit1980 commented 6 years ago

I've just tried the nosqueeze model. CNTK master branch fails on it, but the model loads ok using the ONNX improvements feature branch we are working on. The feature branch should be merged to master this or early next week.