search

facebookarchive / fb.resnet.torch

Torch implementation of ResNet from http://arxiv.org/abs/1512.03385 and training scripts
Other
2.29k stars 664 forks source link

Random luajit errors #187

Open olaron opened 7 years ago

olaron commented 7 years ago

Hi,

After running the script with this command:

gdb --args bash $(which th) main.lua -dataset cifar10 -nGPU 1 -batchSize 128 -depth 110

I randomly get this error:

 | Epoch: [160][186/391]    Time 0.105  Data 0.000  Err 0.0015  top1   0.000  top5   0.000  
 | Epoch: [160][187/391]    Time 0.104  Data 0.000  Err 0.0035  top1   0.000  top5   0.000  
*** Error in `/home/<user>/torch/install/bin/luajit': free(): invalid next size (fast): 0x00007fff30642670 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.6(+0x777e5)[0x7ffff6efe7e5]
/lib/x86_64-linux-gnu/libc.so.6(+0x8037a)[0x7ffff6f0737a]
/lib/x86_64-linux-gnu/libc.so.6(cfree+0x4c)[0x7ffff6f0b53c]
/home/<user>/torch/install/lib/libTH.so.0(THByteTensor_free+0x39)[0x7ffff5e699d9]
/home/<user>/torch/install/lib/lua/5.1/libtorch.so(+0x5014d)[0x7ffff670714d]
/home/<user>/torch/install/bin/luajit[0x47797a]
/home/<user>/torch/install/bin/luajit[0x4396bd]
/home/<user>/torch/install/bin/luajit[0x43983c]
/home/<user>/torch/install/bin/luajit[0x44dba5]
/home/<user>/torch/install/bin/luajit[0x47797a]
/home/<user>/torch/install/bin/luajit[0x466ca1]
/home/<user>/torch/install/bin/luajit[0x47797a]
/home/<user>/torch/install/bin/luajit(lua_pcall+0x2d)[0x4659ad]
/home/<user>/torch/install/bin/luajit[0x40686f]
/home/<user>/torch/install/bin/luajit[0x47797a]
/home/<user>/torch/install/bin/luajit(lua_cpcall+0x17)[0x465a27]
/home/<user>/torch/install/bin/luajit(main+0x34)[0x404544]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7ffff6ea7830]
/home/<user>/torch/install/bin/luajit(_start+0x29)[0x404619]
======= Memory map: ========
00400000-00492000 r-xp 00000000 08:12 80479090                           /home/<user>/torch/install/bin/luajit
005e0000-00640000 r-xp 00000000 00:00 0 
00691000-00692000 r--p 00091000 08:12 80479090                           /home/<user>/torch/install/bin/luajit
00692000-00693000 rw-p 00092000 08:12 80479090                           /home/<user>/torch/install/bin/luajit
00693000-0bcef000 rw-p 00000000 00:00 0                                  [heap]
0bcf0000-0bd10000 r-xp 00000000 00:00 0 
13210000-13260000 r-xp 00000000 00:00 0 
13a90000-13ac0000 r-xp 00000000 00:00 0 
1a450000-1a4b0000 r-xp 00000000 00:00 0 
1ca60000-1cab0000 r-xp 00000000 00:00 0 
40000000-403c0000 rw-p 00000000 00:00 0 
403c1000-412e2000 rw-p 00000000 00:00 0 
41375000-41435000 rw-p 00000000 00:00 0 
200000000-200200000 rw-s 17055b000 00:06 548                             /dev/nvidiactl
200200000-200400000 ---p 00000000 00:00 0 
200400000-200404000 rw-s 4f40ce000 00:06 548                             /dev/nvidiactl
200404000-200600000 ---p 00000000 00:00 0 
200600000-200a00000 rw-s cefb2f000 00:06 548                             /dev/nvidiactl
200a00000-201600000 ---p 00000000 00:00 0 
201600000-201604000 rw-s cf7299000 00:06 548                             /dev/nvidiactl
201604000-201800000 ---p 00000000 00:00 0 
201800000-201c00000 rw-s 4f7522000 00:06 548                             /dev/nvidiactl
201c00000-202800000 ---p 00000000 00:00 0 
202800000-202804000 rw-s f9d41d000 00:06 548                             /dev/nvidiactl
202804000-202a00000 ---p 00000000 00:00 0 
202a00000-202e00000 rw-s fa0be4000 00:06 548                             /dev/nvidiactl
202e00000-203a00000 ---p 00000000 00:00 0 
203a00000-203a04000 rw-s 8142eb000 00:06 548                             /dev/nvidiactl
203a04000-203c00000 ---p 00000000 00:00 0 
203c00000-204000000 rw-s c9dfcc000 00:06 548                             /dev/nvidiactl
204000000-204c00000 ---p 00000000 00:00 0 
204c00000-204c04000 rw-s 5dcb15000 00:06 548                             /dev/nvidiactl
204c04000-204e00000 ---p 00000000 00:00 0 
204e00000-205200000 rw-s cf1992000 00:06 548                             /dev/nvidiactl
205200000-205e00000 ---p 00000000 00:00 0 
205e00000-205e04000 rw-s e27b51000 00:06 548                             /dev/nvidiactl
205e04000-206000000 ---p 00000000 00:00 0 
206000000-206400000 rw-s 18629e000 00:06 548                             /dev/nvidiactl
206400000-207000000 ---p 00000000 00:00 0 
207000000-207004000 rw-s b513c3000 00:06 548                             /dev/nvidiactl
207004000-207200000 ---p 00000000 00:00 0 
207200000-207600000 rw-s 355958000 00:06 548                             /dev/nvidiactl
207600000-208200000 ---p 00000000 00:00 0 
208200000-208204000 rw-s cbf063000 00:06 548                             /dev/nvidiactl
208204000-208400000 ---p 00000000 00:00 0 
208400000-208800000 rw-s fe9bb0000 00:06 548                             /dev/nvidiactl
208800000-209400000 ---p 00000000 00:00 0 
209400000-209404000 rw-s e27b7e000 00:06 548                             /dev/nvidiactl
209404000-209600000 ---p 00000000 00:00 0 
209600000-209a00000 rw-s c0704e000 00:06 548                             /dev/nvidiactl
209a00000-209a04000 rw-s 997b9a000 00:06 548                             /dev/nvidiactl
209a04000-209c00000 ---p 00000000 00:00 0 
209c00000-20a000000 rw-s 7ca7cf000 00:06 548                             /dev/nvidiactl
20a000000-20a004000 rw-s 436c85000 00:06 548                             /dev/nvidiactl
20a004000-20a200000 ---p 00000000 00:00 0 
20a200000-20a600000 rw-s 997bbb000 00:06 548                             /dev/nvidiactl
20a600000-20a604000 rw-s 2146c9000 00:06 548                             /dev/nvidiactl
20a604000-20a800000 ---p 00000000 00:00 0 
20a800000-20ac00000 rw-s 238b16000 00:06 548                             /dev/nvidiactl
20ac00000-20ac04000 rw-s cf73c1000 00:06 548                             /dev/nvidiactl
20ac04000-20ae00000 ---p 00000000 00:00 0 
20ae00000-20b200000 rw-s 12da49000 00:06 548                             /dev/nvidiactl
20b200000-20b204000 rw-s 31e43d000 00:06 548                             /dev/nvidiactl
20b204000-20b400000 ---p 00000000 00:00 0 
20b400000-20b800000 rw-s 12f4ba000 00:06 548                             /dev/nvidiactl
20b800000-20b804000 rw-s 131082000 00:06 548                             /dev/nvidiactl
20b804000-20ba00000 ---p 00000000 00:00 0 
20ba00000-20be00000 rw-s 5dcbe8000 00:06 548                             /dev/nvidiactl
20be00000-20be04000 rw-s c07143000 00:06 548                             /dev/nvidiactl
20be04000-20c000000 ---p 00000000 00:00 0 
20c000000-20c400000 rw-s ff207f000 00:06 548                             /dev/nvidiactl
20c400000-20c600000 ---p 00000000 00:00 0 
20c600000-20c800000 rw-s cefa21000 00:06 548                             /dev/nvidiactl
20c800000-300200000 ---p 00000000 00:00 0 
10000000000-10204200000 ---p 00000000 00:00 0 
10204200000-10204400000 rw-s 65ddc1000 00:06 548                         /dev/nvidiactl
10204400000-10204600000 rw-s 12ce5c000 00:06 548                         /dev/nvidiactl
10204600000-10204800000 rw-s cf72d1000 00:06 548                         /dev/nvidiactl
10204800000-10204ad6000 rw-s 8d561b000 00:06 548                         /dev/nvidiactl
10204ad6000-1020d800000 ---p 00000000 00:00 0 
1020d800000-1020da00000 rw-s 00000000 00:05 803028                       /dev/zero (deleted)
1020da00000-102a2600000 ---p 00000000 00:00 0 
7ffeac000000-7ffeac021000 rw-p 00000000 00:00 0 
7ffeac021000-7ffeb0000000 ---p 00000000 00:00 0 
7ffeb2113000-7fff49b06000 rw-p 00000000 00:00 0 
7fff49b74000-7fff56439000 rw-p 00000000 00:00 0 
7fff56439000-7fff58000000 ---p 00000000 00:00 0 
7fff58024000-7fff5d70e000 rw-p 00000000 00:00 0 
7fff5d70e000-7fff60000000 ---p 00000000 00:00 0 
7fff60000000-7fff623fb000 rw-p 00000000 00:00 0 
7fff623fb000-7fff64000000 ---p 00000000 00:00 0 
7fff64000000-7fff65708000 rw-p 00000000 00:00 0 
7fff65708000-7fff68000000 ---p 00000000 00:00 0 
7fff680ab000-7fff73e02000 rw-p 00000000 00:00 0 
7fff73e98000-7fff7ac00000 rw-p 00000000 00:00 0 
7fff7ac00000-7fff7ac01000 ---p 00000000 00:00 0 
7fff7ac01000-7fff7b401000 rw-p 00000000 00:00 0 
7fff7b401000-7fff7b402000 ---p 00000000 00:00 0 
7fff7b402000-7fff7bc02000 rw-p 00000000 00:00 0 
7fff7bc02000-7fff7bc03000 ---p 00000000 00:00 0 
7fff7bc03000-7fff7c403000 rw-p 00000000 00:00 0 
7fff7c403000-7fff7c404000 ---p 00000000 00:00 0 
7fff7c404000-7fff7cc04000 rw-p 00000000 00:00 0 
7fff7cc04000-7fff7cc05000 r-xp 00000000 08:12 80480777                   /home/<user>/torch/install/lib/libthreadsmain.so
7fff7cc05000-7fff7ce04000 ---p 00001000 08:12 80480777                   /home/<user>/torch/install/lib/libthreadsmain.so
7fff7ce04000-7fff7ce05000 r--p 00000000 08:12 80480777                   /home/<user>/torch/install/lib/libthreadsmain.so
7fff7ce05000-7fff7ce06000 rw-p 00001000 08:12 80480777                   /home/<user>/torch/install/lib/libthreadsmain.so
7fff7ce06000-7fff7ce5d000 r-xp 00000000 08:12 80480760                   /home/<user>/torch/install/lib/lua/5.1/libimage.so
7fff7ce5d000-7fff7d05d000 ---p 00057000 08:12 80480760                   /home/<user>/torch/install/lib/lua/5.1/libimage.so
7fff7d05d000-7fff7d05e000 r--p 00057000 08:12 80480760                   /home/<user>/torch/install/lib/lua/5.1/libimage.so
7fff7d05e000-7fff7d05f000 rw-p 00058000 08:12 80480760                   /home/<user>/torch/install/lib/lua/5.1/libimage.so
7fff7d05f000-7fffa4021000 rw-p 00000000 00:00 0 
7fffa4021000-7fffa8000000 ---p 00000000 00:00 0 
7fffa80c2000-7fffac021000 rw-p 00000000 00:00 0 
7fffac021000-7fffb0000000 ---p 00000000 00:00 0 
7fffb0089000-7fffb16b1000 rw-p 00000000 00:00 0 
7fffb16b1000-7fffb16b2000 ---p 00000000 00:00 0 
7fffb16b2000-7fffb1eb2000 rw-p 00000000 00:00 0 
7fffb1eb2000-7fffb1eb3000 ---p 00000000 00:00 0 
7fffb1eb3000-7fffb2734000 rw-p 00000000 00:00 0 
7fffb2734000-7fffb7756000 r-xp 00000000 08:12 53353092                   /usr/local/cuda-8.0/lib64/libcudnn.so.5.1.10
7fffb7756000-7fffb7956000 ---p 05022000 08:12 53353092                   /usr/local/cuda-8.0/lib64/libcudnn.so.5.1.10
7fffb7956000-7fffb7978000 rw-p 05022000 08:12 53353092                   /usr/local/cuda-8.0/lib64/libcudnn.so.5.1.10
7fffb7978000-7fffb79a3000 rw-p 00000000 00:00 0 
7fffb79a3000-7fffba1f4000 r-xp 00000000 08:12 80480759                   /home/<user>/torch/install/lib/lua/5.1/libTHCUNN.so
7fffba1f4000-7fffba3f3000 ---p 02851000 08:12 80480759                   /home/<user>/torch/install/lib/lua/5.1/libTHCUNN.so
7fffba3f3000-7fffba3ff000 r--p 02850000 08:12 80480759                   /home/<user>/torch/install/lib/lua/5.1/libTHCUNN.so
7fffba3ff000-7fffba401000 rw-p 0285c000 08:12 80480759                   /home/<user>/torch/install/lib/lua/5.1/libTHCUNN.so
7fffba401000-7fffba415000 rw-p 00000000 00:00 0 
7fffba415000-7fffba416000 ---p 00000000 00:00 0 
7fffba416000-7fffbac16000 rw-p 00000000 00:00 0 
7fffbac16000-7fffc0c16000 ---p 00000000 00:00 0 
7fffc0c16000-7fffc0c59000 r-xp 00000000 08:12 47188617                   /usr/lib/nvidia-381/libnvidia-fatbinaryloader.so.381.22
7fffc0c59000-7fffc0e58000 ---p 00043000 08:12 47188617                   /usr/lib/nvidia-381/libnvidia-fatbinaryloader.so.381.22
7fffc0e58000-7fffc0e62000 rw-p 00042000 08:12 47188617                   /usr/lib/nvidia-381/libnvidia-fatbinaryloader.so.381.22
7fffc0e62000-7fffc0e68000 rw-p 00000000 00:00 0 
7fffc0e68000-7fffc1591000 r-xp 00000000 08:12 46934708                   /usr/lib/x86_64-linux-gnu/libcuda.so.381.22
7fffc1591000-7fffc1790000 ---p 00729000 08:12 46934708                   /usr/lib/x86_64-linux-gnu/libcuda.so.381.22
7fffc1790000-7fffc18af000 rw-p 00728000 08:12 46934708                   /usr/lib/x86_64-linux-gnu/libcuda.so.381.22
7fffc18af000-7fffc18bd000 rw-p 00000000 00:00 0 
7fffc18bd000-7fffc1a2f000 r-xp 00000000 08:12 46932730                   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7fffc1a2f000-7fffc1c2f000 ---p 00172000 08:12 46932730                   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7fffc1c2f000-7fffc1c39000 r--p 00172000 08:12 46932730                   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7fffc1c39000-7fffc1c3b000 rw-p 0017c000 08:12 46932730                   /usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.21
7fffc1c3b000-7fffc1c3f000 rw-p 00000000 00:00 0 
7fffc1c3f000-7fffc4530000 r-xp 00000000 08:12 53351998                   /usr/local/cuda-8.0/lib64/libcusparse.so.8.0.61
7fffc4530000-7fffc4730000 ---p 028f1000 08:12 53351998                   /usr/local/cuda-8.0/lib64/libcusparse.so.8.0.61
7fffc4730000-7fffc4749000 rw-p 028f1000 08:12 53351998                   /usr/local/cuda-8.0/lib64/libcusparse.so.8.0.61
7fffc4749000-7fffc475a000 rw-p 00000000 00:00 0 
7fffc475a000-7fffc7572000 r-xp 00000000 08:12 53353087                   /usr/local/cuda-8.0/lib64/libcublas.so.8.0.88
7fffc7572000-7fffc7772000 ---p 02e18000 08:12 53353087                   /usr/local/cuda-8.0/lib64/libcublas.so.8.0.88
7fffc7772000-7fffc7790000 rw-p 02e18000 08:12 53353087                   /usr/local/cuda-8.0/lib64/libcublas.so.8.0.88
7fffc7790000-7fffc779f000 rw-p 00000000 00:00 0 
7fffc779f000-7fffc7801000 r-xp 00000000 08:12 53352051                   /usr/local/cuda-8.0/lib64/libcudart.so.8.0.61
7fffc7801000-7fffc7a01000 ---p 00062000 08:12 53352051                   /usr/local/cuda-8.0/lib64/libcudart.so.8.0.61
7fffc7a01000-7fffc7a04000 rw-p 00062000 08:12 53352051                   /usr/local/cuda-8.0/lib64/libcudart.so.8.0.61
7fffc7a04000-7fffc7a05000 rw-p 00000000 00:00 0 
7fffc7a05000-7fffd2f0a000 r-xp 00000000 08:12 80480763                   /home/<user>/torch/install/lib/libTHC.so.0
7fffd2f0a000-7fffd310a000 ---p 0b505000 08:12 80480763                   /home/<user>/torch/install/lib/libTHC.so.0
7fffd310a000-7fffd3146000 r--p 0b505000 08:12 80480763                   /home/<user>/torch/install/lib/libTHC.so.0
7fffd3146000-7fffd315a000 rw-p 0b541000 08:12 80480763                   /home/<user>/torch/install/lib/libTHC.so.0
7fffd315a000-7fffd3170000 rw-p 00000000 00:00 0 
7fffd3170000-7fffd32d9000 r-xp 00000000 08:12 80480765                   /home/<user>/torch/install/lib/lua/5.1/libcutorch.so
7fffd32d9000-7fffd34d9000 ---p 00169000 08:12 80480765                   /home/<user>/torch/install/lib/lua/5.1/libcutorch.so
7fffd34d9000-7fffd34e1000 r--p 00169000 08:12 80480765                   /home/<user>/torch/install/lib/lua/5.1/libcutorch.so
7fffd34e1000-7fffd34e6000 rw-p 00171000 08:12 80480765                   /home/<user>/torch/install/lib/lua/5.1/libcutorch.so
7fffd34e6000-7fffd34ec000 r-xp 00000000 08:12 80480779                   /home/<user>/torch/install/lib/lua/5.1/libthreads.so
7fffd34ec000-7fffd36eb000 ---p 00006000 08:12 80480779                   /home/<user>/torch/install/lib/lua/5.1/libthreads.so
7fffd36eb000-7fffd36ec000 r--p 00005000 08:12 80480779                   /home/<user>/torch/install/lib/lua/5.1/libthreads.so
7fffd36ec000-7fffd36ed000 rw-p 00006000 08:12 80480779                   /home/<user>/torch/install/lib/lua/5.1/libthreads.so
7fffd36ed000-7fffd3808000 r-xp 00000000 08:12 80480755                   /home/<user>/torch/install/lib/lua/5.1/libTHNN.so
7fffd3808000-7fffd3a07000 ---p 0011b000 08:12 80480755                   /home/<user>/torch/install/lib/lua/5.1/libTHNN.so
7fffd3a07000-7fffd3a08000 r--p 0011a000 08:12 80480755                   /home/<user>/torch/install/lib/lua/5.1/libTHNN.so
7fffd3a08000-7fffd3a09000 rw-p 0011b000 08:12 80480755                   /home/<user>/torch/install/lib/lua/5.1/libTHNN.so
7fffd3a09000-7fffd3ce1000 r--p 00000000 08:12 46925246                   /usr/lib/locale/locale-archive
7fffd3ce1000-7fffd3ce4000 r-xp 00000000 08:12 80479708                   /home/<user>/torch/install/lib/lua/5.1/readline.so
7fffd3ce4000-7fffd3ee3000 ---p 00003000 08:12 80479708                   /home/<user>/torch/install/lib/lua/5.1/readline.so
7fffd3ee3000-7fffd3ee4000 r--p 00002000 08:12 80479708                   /home/<user>/torch/install/lib/lua/5.1/readline.so
7fffd3ee4000-7fffd3ee5000 rw-p 00003000 08:12 80479708                   /home/<user>/torch/install/lib/lua/5.1/readline.so
7fffd3ee5000-7fffd3ee6000 r-xp 00000000 08:12 80480754                   /home/<user>/torch/install/lib/lua/5.1/treplutils.so
7fffd3ee6000-7fffd40e5000 ---p 00001000 08:12 80480754                   /home/<user>/torch/install/lib/lua/5.1/treplutils.so
7fffd40e5000-7fffd40e6000 r--p 00000000 08:12 80480754                   /home/<user>/torch/install/lib/lua/5.1/treplutils.so
7fffd40e6000-7fffd40e7000 rw-p 00001000 08:12 80480754                   /home/<user>/torch/install/lib/lua/5.1/treplutils.so
7fffd40e7000-7fffd40f4000 r-xp 00000000 08:12 80479442                   /home/<user>/torch/install/lib/lua/5.1/libsundown.so
7fffd40f4000-7fffd42f4000 ---p 0000d000 08:12 80479442                   /home/<user>/torch/install/lib/lua/5.1/libsundown.so
7fffd42f4000-7fffd42f5000 r--p 0000d000 08:12 80479442                   /home/<user>/torch/install/lib/lua/5.1/libsundown.so
7fffd42f5000-7fffd42f6000 rw-p 0000e000 08:12 80479442                   /home/<user>/torch/install/lib/lua/5.1/libsundown.so
7fffd42f6000-7fffd42f7000 r-xp 00000000 08:12 80479707                   /home/<user>/torch/install/lib/lua/5.1/libsys.so
7fffd42f7000-7fffd44f6000 ---p 00001000 08:12 80479707                   /home/<user>/torch/install/lib/lua/5.1/libsys.so
7fffd44f6000-7fffd44f7000 r--p 00000000 08:12 80479707                   /home/<user>/torch/install/lib/lua/5.1/libsys.so
7fffd44f7000-7fffd44f8000 rw-p 00001000 08:12 80479707                   /home/<user>/torch/install/lib/lua/5.1/libsys.so
7fffd44f8000-7ffff44f8000 rw-p 00000000 00:00 0 
7ffff44f8000-7ffff4536000 r-xp 00000000 08:12 46932604                   /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0
7ffff4536000-7ffff4735000 ---p 0003e000 08:12 46932604                   /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0
7ffff4735000-7ffff4736000 r--p 0003d000 08:12 46932604                   /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0
7ffff4736000-7ffff4737000 rw-p 0003e000 08:12 46932604                   /usr/lib/x86_64-linux-gnu/libquadmath.so.0.0.0
7ffff4737000-7ffff4860000 r-xp 00000000 08:12 46934443                   /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0
7ffff4860000-7ffff4a5f000 ---p 00129000 08:12 46934443                   /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0
7ffff4a5f000-7ffff4a60000 r--p 00128000 08:12 46934443                   /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0
7ffff4a60000-7ffff4a62000 rw-p 00129000 08:12 46934443                   /usr/lib/x86_64-linux-gnu/libgfortran.so.3.0.0
7ffff4a62000-7ffff4a83000 r-xp 00000000 08:12 46932114                   /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0
7ffff4a83000-7ffff4c82000 ---p 00021000 08:12 46932114                   /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0
7ffff4c82000-7ffff4c83000 r--p 00020000 08:12 46932114                   /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0
7ffff4c83000-7ffff4c84000 rw-p 00021000 08:12 46932114                   /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0
7ffff4c84000-7ffff4c8b000 r-xp 00000000 08:12 18350171                   /lib/x86_64-linux-gnu/librt-2.23.so
7ffff4c8b000-7ffff4e8a000 ---p 00007000 08:12 18350171                   /lib/x86_64-linux-gnu/librt-2.23.so
7ffff4e8a000-7ffff4e8b000 r--p 00006000 08:12 18350171                   /lib/x86_64-linux-gnu/librt-2.23.so
7ffff4e8b000-7ffff4e8c000 rw-p 00007000 08:12 18350171                   /lib/x86_64-linux-gnu/librt-2.23.so
7ffff4e8c000-7ffff5bb0000 r-xp 00000000 08:12 81526802                   /opt/OpenBLAS/lib/libopenblas_haswellp-r0.2.20.dev.so
7ffff5bb0000-7ffff5db0000 ---p 00d24000 08:12 81526802                   /opt/OpenBLAS/lib/libopenblas_haswellp-r0.2.20.dev.so
7ffff5db0000-7ffff5db6000 r--p 00d24000 08:12 81526802                   /opt/OpenBLAS/lib/libopenblas_haswellp-r0.2.20.dev.so
7ffff5db6000-7ffff5dc0000 rw-p 00d2a000 08:12 81526802                   /opt/OpenBLAS/lib/libopenblas_haswellp-r0.2.20.dev.so
7ffff5dc0000-7ffff5e1a000 rw-p 00000000 00:00 0 
7ffff5e1a000-7ffff62a9000 r-xp 00000000 08:12 80479669                   /home/<user>/torch/install/lib/libTH.so.0
7ffff62a9000-7ffff64a9000 ---p 0048f000 08:12 80479669                   /home/<user>/torch/install/lib/libTH.so.0
7ffff64a9000-7ffff64aa000 r--p 0048f000 08:12 80479669                   /home/<user>/torch/install/lib/libTH.so.0
7ffff64aa000-7ffff64ad000 rw-p 00490000 08:12 80479669                   /home/<user>/torch/install/lib/libTH.so.0
7ffff64ad000-7ffff64b6000 r-xp 00000000 08:12 80479672                   /home/<user>/torch/install/lib/libluaT.so.0
7ffff64b6000-7ffff66b5000 ---p 00009000 08:12 80479672                   /home/<user>/torch/install/lib/libluaT.so.0
7ffff66b5000-7ffff66b6000 r--p 00008000 08:12 80479672                   /home/<user>/torch/install/lib/libluaT.so.0
7ffff66b6000-7ffff66b7000 rw-p 00009000 08:12 80479672                   /home/<user>/torch/install/lib/libluaT.so.0
7ffff66b7000-7ffff684d000 r-xp 00000000 08:12 80479531                   /home/<user>/torch/install/lib/lua/5.1/libtorch.so
7ffff684d000-7ffff6a4c000 ---p 00196000 08:12 80479531                   /home/<user>/torch/install/lib/lua/5.1/libtorch.so
7ffff6a4c000-7ffff6a55000 r--p 00195000 08:12 80479531                   /home/<user>/torch/install/lib/lua/5.1/libtorch.so
7ffff6a55000-7ffff6a59000 rw-p 0019e000 08:12 80479531                   /home/<user>/torch/install/lib/lua/5.1/libtorch.so
7ffff6a59000-7ffff6a5d000 r-xp 00000000 08:12 80479475                   /home/<user>/torch/install/lib/lua/5.1/libpaths.so
7ffff6a5d000-7ffff6c5c000 ---p 00004000 08:12 80479475                   /home/<user>/torch/install/lib/lua/5.1/libpaths.so
7ffff6c5c000-7ffff6c5d000 r--p 00003000 08:12 80479475                   /home/<user>/torch/install/lib/lua/5.1/libpaths.so
7ffff6c5d000-7ffff6c5e000 rw-p 00004000 08:12 80479475                   /home/<user>/torch/install/lib/lua/5.1/libpaths.so
7ffff6c5e000-7ffff6c83000 r-xp 00000000 08:12 18354745                   /lib/x86_64-linux-gnu/libtinfo.so.5.9
7ffff6c83000-7ffff6e82000 ---p 00025000 08:12 18354745                   /lib/x86_64-linux-gnu/libtinfo.so.5.9
7ffff6e82000-7ffff6e86000 r--p 00024000 08:12 18354745                   /lib/x86_64-linux-gnu/libtinfo.so.5.9
7ffff6e86000-7ffff6e87000 rw-p 00028000 08:12 18354745                   /lib/x86_64-linux-gnu/libtinfo.so.5.9
7ffff6e87000-7ffff7047000 r-xp 00000000 08:12 18354594                   /lib/x86_64-linux-gnu/libc-2.23.so
7ffff7047000-7ffff7247000 ---p 001c0000 08:12 18354594                   /lib/x86_64-linux-gnu/libc-2.23.so
7ffff7247000-7ffff724b000 r--p 001c0000 08:12 18354594                   /lib/x86_64-linux-gnu/libc-2.23.so
7ffff724b000-7ffff724d000 rw-p 001c4000 08:12 18354594                   /lib/x86_64-linux-gnu/libc-2.23.so
7ffff724d000-7ffff7251000 rw-p 00000000 00:00 0 
7ffff7251000-7ffff7267000 r-xp 00000000 08:12 18354611                   /lib/x86_64-linux-gnu/libgcc_s.so.1
7ffff7267000-7ffff7466000 ---p 00016000 08:12 18354611                   /lib/x86_64-linux-gnu/libgcc_s.so.1
7ffff7466000-7ffff7467000 rw-p 00015000 08:12 18354611                   /lib/x86_64-linux-gnu/libgcc_s.so.1
7ffff7467000-7ffff747f000 r-xp 00000000 08:12 18350168                   /lib/x86_64-linux-gnu/libpthread-2.23.so
7ffff747f000-7ffff767e000 ---p 00018000 08:12 18350168                   /lib/x86_64-linux-gnu/libpthread-2.23.so
7ffff767e000-7ffff767f000 r--p 00017000 08:12 18350168                   /lib/x86_64-linux-gnu/libpthread-2.23.so
7ffff767f000-7ffff7680000 rw-p 00018000 08:12 18350168                   /lib/x86_64-linux-gnu/libpthread-2.23.so
7ffff7680000-7ffff7684000 rw-p 00000000 00:00 0 
7ffff7684000-7ffff778c000 r-xp 00000000 08:12 18354528                   /lib/x86_64-linux-gnu/libm-2.23.so
7ffff778c000-7ffff798b000 ---p 00108000 08:12 18354528                   /lib/x86_64-linux-gnu/libm-2.23.so
7ffff798b000-7ffff798c000 r--p 00107000 08:12 18354528                   /lib/x86_64-linux-gnu/libm-2.23.so
7ffff798c000-7ffff798d000 rw-p 00108000 08:12 18354528                   /lib/x86_64-linux-gnu/libm-2.23.so
7ffff798d000-7ffff7990000 r-xp 00000000 08:12 18350174                   /lib/x86_64-linux-gnu/libdl-2.23.so
7ffff7990000-7ffff7b8f000 ---p 00003000 08:12 18350174                   /lib/x86_64-linux-gnu/libdl-2.23.so
7ffff7b8f000-7ffff7b90000 r--p 00002000 08:12 18350174                   /lib/x86_64-linux-gnu/libdl-2.23.so
7ffff7b90000-7ffff7b91000 rw-p 00003000 08:12 18350174                   /lib/x86_64-linux-gnu/libdl-2.23.so
7ffff7b91000-7ffff7bce000 r-xp 00000000 08:12 18354724                   /lib/x86_64-linux-gnu/libreadline.so.6.3
7ffff7bce000-7ffff7dce000 ---p 0003d000 08:12 18354724                   /lib/x86_64-linux-gnu/libreadline.so.6.3
7ffff7dce000-7ffff7dd0000 r--p 0003d000 08:12 18354724                   /lib/x86_64-linux-gnu/libreadline.so.6.3
7ffff7dd0000-7ffff7dd6000 rw-p 0003f000 08:12 18354724                   /lib/x86_64-linux-gnu/libreadline.so.6.3
7ffff7dd6000-7ffff7dd7000 rw-p 00000000 00:00 0 
7ffff7dd7000-7ffff7dfd000 r-xp 00000000 08:12 18350163                   /lib/x86_64-linux-gnu/ld-2.23.so
7ffff7e44000-7ffff7e45000 rw-p 00000000 00:00 0 
7ffff7e45000-7ffff7e46000 rw-s cfdf6000 00:06 549                        /dev/nvidia0
7ffff7e46000-7ffff7e47000 rw-s c0e109000 00:06 548                       /dev/nvidiactl
7ffff7e47000-7ffff7e48000 rw-s cfdf6000 00:06 549                        /dev/nvidia0
7ffff7e48000-7ffff7e49000 rw-s c07142000 00:06 548                       /dev/nvidiactl
7ffff7e49000-7ffff7fb9000 rw-p 00000000 00:00 0 
7ffff7fb9000-7ffff7fba000 rw-s cfdf6000 00:06 549                        /dev/nvidia0
7ffff7fba000-7ffff7fbb000 rw-s 131080000 00:06 548                       /dev/nvidiactl
7ffff7fbb000-7ffff7fbc000 rw-s cfdf6000 00:06 549                        /dev/nvidia0
7ffff7fbc000-7ffff7fbd000 rw-s 31e43b000 00:06 548                       /dev/nvidiactl
7ffff7fbd000-7ffff7fbe000 rw-s cfdf6000 00:06 549                        /dev/nvidia0
7ffff7fbe000-7ffff7fbf000 rw-s cf73c0000 00:06 548                       /dev/nvidiactl
7ffff7fbf000-7ffff7fc0000 rw-s cfdf6000 00:06 549                        /dev/nvidia0
7ffff7fc0000-7ffff7fc1000 rw-s 2146c8000 00:06 548                       /dev/nvidiactl
7ffff7fc1000-7ffff7fc2000 rw-s cfdf6000 00:06 549                        /dev/nvidia0
7ffff7fc2000-7ffff7fc3000 rw-s 436c83000 00:06 548                       /dev/nvidiactl
7ffff7fc3000-7ffff7fc4000 rw-s cfdf6000 00:06 549                        /dev/nvidia0
7ffff7fc4000-7ffff7fc5000 rw-s 997b98000 00:06 548                       /dev/nvidiactl
7ffff7fc5000-7ffff7fc6000 rw-s cfdf5000 00:06 549                        /dev/nvidia0
7ffff7fc6000-7ffff7fc7000 rw-s e27b7a000 00:06 548                       /dev/nvidiactl
7ffff7fc7000-7ffff7fc8000 rw-s cfdf5000 00:06 549                        /dev/nvidia0
7ffff7fc8000-7ffff7fc9000 rw-s cbf062000 00:06 548                       /dev/nvidiactl
7ffff7fc9000-7ffff7fca000 rw-s cfdf5000 00:06 549                        /dev/nvidia0
7ffff7fca000-7ffff7fcb000 rw-s b513c1000 00:06 548                       /dev/nvidiactl
7ffff7fcb000-7ffff7fcc000 rw-s cfdf5000 00:06 549                        /dev/nvidia0
7ffff7fcc000-7ffff7fcd000 rw-s e27b50000 00:06 548                       /dev/nvidiactl
7ffff7fcd000-7ffff7fce000 rw-s cfdf5000 00:06 549                        /dev/nvidia0
7ffff7fce000-7ffff7fcf000 rw-s 5dcb14000 00:06 548                       /dev/nvidiactl
7ffff7fcf000-7ffff7fd0000 rw-s cfdf5000 00:06 549                        /dev/nvidia0
7ffff7fd0000-7ffff7fd1000 rw-s 8142ea000 00:06 548                       /dev/nvidiactl
7ffff7fd1000-7ffff7fd2000 rw-s cfdf5000 00:06 549                        /dev/nvidia0
7ffff7fd2000-7ffff7fd3000 rw-s f9d41c000 00:06 548                       /dev/nvidiactl
7ffff7fd3000-7ffff7fd4000 rw-s cfdf5000 00:06 549                        /dev/nvidia0
7ffff7fd4000-7ffff7fd5000 rw-s cf7298000 00:06 548                       /dev/nvidiactl
7ffff7fd5000-7ffff7ff8000 rw-p 00000000 00:00 0 
7ffff7ff8000-7ffff7ffa000 r--p 00000000 00:00 0                          [vvar]
7ffff7ffa000-7ffff7ffc000 r-xp 00000000 00:00 0                          [vdso]
7ffff7ffc000-7ffff7ffd000 r--p 00025000 08:12 18350163                   /lib/x86_64-linux-gnu/ld-2.23.so
7ffff7ffd000-7ffff7ffe000 rw-p 000260
Thread 1 "luajit" received signal SIGABRT, Aborted.
0x00007ffff6ebc428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
54  ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  0x00007ffff6ebc428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1  0x00007ffff6ebe02a in __GI_abort () at abort.c:89
#2  0x00007ffff6efe7ea in __libc_message (do_abort=do_abort@entry=2, fmt=fmt@entry=0x7ffff7017e98 "*** Error in `%s': %s: 0x%s ***\n")
    at ../sysdeps/posix/libc_fatal.c:175
#3  0x00007ffff6f0737a in malloc_printerr (ar_ptr=<optimised out>, ptr=<optimised out>, str=0x7ffff7017f10 "free(): invalid next size (fast)", action=3)
    at malloc.c:5006
#4  _int_free (av=<optimised out>, p=<optimised out>, have_lock=0) at malloc.c:3867
#5  0x00007ffff6f0b53c in __GI___libc_free (mem=<optimised out>) at malloc.c:2968
#6  0x00007ffff5e699d9 in THByteTensor_free () from /home/<user>/torch/install/lib/libTH.so.0
#7  0x00007ffff670714d in torch_ByteTensor_free () from /home/<user>/torch/install/lib/lua/5.1/libtorch.so
#8  0x000000000047797a in lj_BC_FUNCC ()
#9  0x00000000004396bd in gc_onestep ()
#10 0x000000000043983c in lj_gc_step ()
#11 0x000000000044dba5 in lj_cf_ffi_meta___unm ()
#12 0x000000000047797a in lj_BC_FUNCC ()
#13 0x0000000000466ca1 in lj_cf_dofile ()
#14 0x000000000047797a in lj_BC_FUNCC ()
#15 0x00000000004659ad in lua_pcall ()
#16 0x000000000040686f in pmain ()
#17 0x000000000047797a in lj_BC_FUNCC ()
#18 0x0000000000465a27 in lua_cpcall ()
#19 0x0000000000404544 in main ()
(gdb)

Or this error:

 | Test: [38][40/79]    Time 0.027  Data 0.000  top1  12.500 ( 14.805)  top5   0.000 (  0.586)  
 | Test: [38][41/79]    Time 0.027  Data 0.000  top1  17.969 ( 14.882)  top5   0.781 (  0.591)  

Thread 8 "luajit" received signal SIGBUS, Bus error.
[Switching to Thread 0x7fff7a822700 (LWP 15795)]
malloc_consolidate (av=av@entry=0x7fff54000020) at malloc.c:4169
4169    malloc.c: No such file or directory.
(gdb) bt
#0  malloc_consolidate (av=av@entry=0x7fff54000020) at malloc.c:4169
#1  0x00007ffff6f08cde in _int_malloc (av=av@entry=0x7fff54000020, bytes=bytes@entry=12400) at malloc.c:3450
#2  0x00007ffff6f09c0a in _int_memalign (av=av@entry=0x7fff54000020, alignment=alignment@entry=64, bytes=bytes@entry=12288) at malloc.c:4421
#3  0x00007ffff6f0e71d in _mid_memalign (address=<optimised out>, bytes=12288, alignment=64) at malloc.c:3120
#4  __posix_memalign (memptr=0x7fff7a821dd8, alignment=<optimised out>, size=12288) at malloc.c:5028
#5  0x00007ffff5e4922c in THAlloc () from /home/<user>/torch/install/lib/libTH.so.0
#6  0x00007ffff5e4ab98 in THFloatStorage_resize () from /home/<user>/torch/install/lib/libTH.so.0
#7  0x00007ffff67267e5 in torch_FloatTensor_resize () from /home/<user>/torch/install/lib/lua/5.1/libtorch.so
#8  0x000000000047797a in lj_BC_FUNCC ()
#9  0x00000000004659ad in lua_pcall ()
#10 0x00007fff7c0269a8 in THThread_main () from /home/<user>/torch/install/lib/libthreadsmain.so
#11 0x00007ffff746e6ba in start_thread (arg=0x7fff7a822700) at pthread_create.c:333
#12 0x00007ffff6f8e3dd in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:109
(gdb)

Or this:

 | Epoch: [67][226/391]    Time 0.100  Data 0.000  Err 0.2143  top1   9.375  top5   0.000   
 | Epoch: [67][227/391]    Time 0.100  Data 0.000  Err 0.1919  top1   7.031  top5   0.000   

Thread 1 "luajit" received signal SIGSEGV, Segmentation fault.
0x000000000040bf60 in lj_alloc_malloc ()
(gdb) bt
#0  0x000000000040bf60 in lj_alloc_malloc ()
#1  0x000000000042988a in lj_mem_newgco ()
#2  0x00000000144c7e6f in ?? ()
#3  0x0000000040000378 in ?? ()
#4  0x000000000047797a in lj_BC_FUNCC ()
#5  0x0000000000466ca1 in lj_cf_dofile ()
#6  0x000000000047797a in lj_BC_FUNCC ()
#7  0x00000000004659ad in lua_pcall ()
#8  0x000000000040686f in pmain ()
#9  0x000000000047797a in lj_BC_FUNCC ()
#10 0x0000000000465a27 in lua_cpcall ()
#11 0x0000000000404544 in main ()
(gdb)

My configuration: Ubuntu 16.04 LTS 64-bit GeForce GTX 1080 Ti (Driver Version: 381.22) CUDA 8.0 cuDNN v5.1 Torch7 (https://github.com/torch/distro/commit/5beb83c46e91abd273c192a3fa782b62217072a6)