search

CMU-Perceptual-Computing-Lab / caffe_rtpose

Realtime C++ code for multi-person pose estimation
Other
356 stars 207 forks source link

It seems like out of memory #34

Closed lfxxxxx closed 7 years ago

lfxxxxx commented 7 years ago

[lfx@gpu-com caffe_rtpose-master]$ ./build/examples/rtpose/rtpose.bin --image_dir ~/caffe_rtpose-master/image/ ./build/examples/rtpose/rtpose.bin: /home/lfx/tools/anaconda2/lib/libtiff.so.5: no version information available (required by /usr/local/lib/libopencv_imgcodecs.so.3.1) F0402 10:24:15.074151 25048 syncedmem.cpp:56] Check failed: error == cudaSuccess (2 vs. 0) out of memory Check failure stack trace: @ 0x7f7d4a812e6d (unknown) @ 0x7f7d4a814ced (unknown) @ 0x7f7d4a812a5c (unknown) @ 0x7f7d4a81563e (unknown) @ 0x7f7d519eaed2 caffe::SyncedMemory::to_gpu() @ 0x7f7d519ea249 caffe::SyncedMemory::mutable_gpu_data() @ 0x7f7d51863e62 caffe::Blob<>::mutable_gpu_data() @ 0x7f7d51a60e7f caffe::CuDNNConvolutionLayer<>::Forward_gpu() @ 0x7f7d519b2f2b caffe::Net<>::ForwardFromTo() @ 0x40bdff warmup() @ 0x411e3b processFrame() @ 0x7f7d3eb95dc5 start_thread @ 0x7f7d3e8c473d __clone Aborted (core dumped)

[lfx@gpu-com caffe_rtpose-master]$ nvidia-smi Sun Apr 2 10:25:18 2017
+-----------------------------------------------------------------------------+ | NVIDIA-SMI 375.26 Driver Version: 375.26 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===============================+======================+======================| | 0 GeForce GT 730 Off | 0000:01:00.0 N/A | N/A | | 40% 59C P8 N/A / N/A | 2MiB / 978MiB | N/A Default | +-------------------------------+----------------------+----------------------+ | 1 GeForce GTX TIT... Off | 0000:06:00.0 Off | N/A | | 53% 83C P2 184W / 250W | 4192MiB / 12206MiB | 97% Default | +-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |=============================================================================| | 0 Not Supported | | 1 12602 C python 139MiB | | 1 22669 C python 139MiB | | 1 23028 C /home/xhq/caffe/build/tools/caffe 3903MiB | +-----------------------------------------------------------------------------+

I have no idea why still have the error "out of memory" even test in just 1 image. [lfx@gpu-com caffe_rtpose-master]$ ls image image1.jpg

gineshidalgo99 commented 7 years ago

Hi, you need around 2GB of free memory on your GPU to run our code. You can check that with the bash command: nvidia-smi. Are you sure you are using the GPU with 12GB instead of the one of only 1GB? Another thought is that you compiled without cuDNN. Without cuDNN, it requires more than 12GB of GPU memory. Let us know if any of this was your problem, thanks! Otherwise, re-open this issue please.

lfxxxxx commented 7 years ago

I have compiled rtpose-caffe with cuDNN. (I have set USE_CUDNN:=1), but still have this error

[lfx@gpu-com rtpose-caffe]$ .build_release/examples/rtpose/rtpose.bin --image_dir image/ F0407 10:27:17.102958 1129 syncedmem.cpp:64] Check failed: error == cudaSuccess (2 vs. 0) out of memory Check failure stack trace: @ 0x7efeaa736e6d (unknown) @ 0x7efeaa738ced (unknown) @ 0x7efeaa736a5c (unknown) @ 0x7efeaa73963e (unknown) @ 0x7efeb1905912 caffe::SyncedMemory::to_gpu() @ 0x7efeb1904c79 caffe::SyncedMemory::gpu_data() @ 0x7efeb177e762 caffe::Blob<>::gpu_data() @ 0x7efeb197b9d3 caffe::CuDNNConvolutionLayer<>::Forward_gpu() @ 0x7efeb18cdb8b caffe::Net<>::ForwardFromTo() @ 0x40bcff warmup() @ 0x411cbb processFrame() @ 0x7efe9ed99dc5 start_thread @ 0x7efe9eac873d __clone Aborted (core dumped)

gineshidalgo99 commented 7 years ago

I will answer in #37. Thanks