search

YipengHu / label-reg

(This repo is no longer up-to-date. Any updates will be at https://github.com/DeepRegNet/DeepReg/) A demo of the re-factored label-driven registration code, based on "Weakly-supervised convolutional neural networks for multimodal image registration"
Apache License 2.0
118 stars 32 forks source link

image size #9

Closed blossomzx closed 6 years ago

blossomzx commented 6 years ago

I used my image data(the sizes of the moving images and the fixed images are all 117x100x255 ) to train the network, and occurred the following problem:

ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[1,117,100,255,64] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc [[Node: local_up_0/additive_upsampling/resize_volume/transpose_1 = Transpose[T=DT_FLOAT, Tperm=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](local_up_0/additive_upsampling/resize_volume/Reshape_3, local_up_0/additive_upsampling/resize_volume/transpose/perm)]] Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

This problem still existed even I changed the minibatch_size to 1. Do I have to change the size of the images?

YipengHu commented 6 years ago

Hi, it looks like a memory problem to me. If possible, resampling your image, by a factor between 0.5 and 1, should work in your case. Mini batch size of 64 is definitely too big, I'd normally use 2 - 8 pairs of images for a mini batch. Of course, it depends the size of the gpu you have.

On Mon, Oct 8, 2018, 15:36 Yang notifications@github.com wrote:

I used my image data(the sizes of the moving images and the fixed images are all 117x100x255 ) to train the network, and occurred the following problem:

ResourceExhaustedError (see above for traceback): OOM when allocating tensor with shape[1,117,100,255,64] and type float on /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc [[Node: local_up_0/additive_upsampling/resize_volume/transpose_1 = TransposeT=DT_FLOAT, Tperm=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"]] Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info.

This problem still existed even I changed the minibatch_size to 1. Do I have to change the size of the images?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/YipengHu/label-reg/issues/9, or mute the thread https://github.com/notifications/unsubscribe-auth/ACd9_eyYZU_B7T1ZTydJ50k29HsZkhnNks5ui2LwgaJpZM4XNErX .

blossomzx commented 6 years ago

Thank you very much.