Closed qianjiangcn closed 5 years ago
tomvars
commented
5 years ago This suggests your GPU doesn't have enough memory to load the 2D-unet. You could try reducing the batch size and making sure that the memory on your NVIDIA GPU is cleared before running NiftyNet using nvidia-smi.
qianjiangcn
commented
5 years ago Thank you very much, I resolved this problem!
1160914483
commented
5 years ago Hello, I have encountered the following problems when running Unet_2D:
Valueerror: Negative dimension size caused by subtracting 3 from 2 for 'worker_0/UNet2D/TwoConv_5/convprelu/conv/conv'(op:'conv2D') with input shape: [1, 2, 2, 1024], [3, 3 , 1024, 512]
Have you encountered the same reason?
wyli
commented
5 years ago Looks like your input window size is too small to use unet, check out this:https://github.com/NifTK/NiftyNet/blob/dev/demos/unet/U-Net_Demo.ipynb
On 4 Mar 2019, at 01:18, 1160914483 notifications@github.com<mailto:notifications@github.com> wrote:
Hello, I have encountered the following problems when running Unet_2D:
Valueerror: Negative dimension size caused by subtracting 3 from 2 for 'worker_0/UNet2D/TwoConv_5/convprelu/conv/conv'(op:'conv2D') with input shape: [1, 2, 2, 1024], [3, 3 , 1024, 512]
Have you encountered the same reason?
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/NifTK/NiftyNet/issues/260#issuecomment-469087565, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AAywXP4b_8FZiqoWlum8Kc35x2uNZI5_ks5vTHRFgaJpZM4X7SVy.
1160914483
commented
5 years ago @wyli Thank you for your reply, I solved this problem with your suggestion.
But I have another problem that I can't solve.When I was running 2D_Unet, the LOSS value could not be lowered and it was stable around 0.75.(My data size is (240, 240, 155)
The following is my configuration file parameters, and I very much hope that you can give a way to solve this problem.
[T1] path_to_search = ../Brats17trainData filename_contains = _t1 filename_not_contains = t1ce spatial_window_size = (236, 236) interp_order = 1 axcodes=(A, R, S)
[T2] path_to_search = ../Brats17trainData filename_contains = _t2 filename_not_contains = spatial_window_size = (236, 236) interp_order = 1 axcodes=(A, R, S)
[T1ce] path_to_search = ../Brats17trainData filename_contains = _t1ce filename_not_contains = spatial_window_size = (236, 236) interp_order = 1 axcodes=(A, R, S)
[Flair] path_to_search = ../Brats17trainData filename_contains = _flair filename_not_contains = spatial_window_size = (236, 236) interp_order = 1 axcodes=(A, R, S)
[label] path_to_search = ../Brats17trainData/label filename_contains = _seg filename_not_contains = spatial_window_size = (52, 52) interp_order = 0 axcodes=(A, R, S)
############################## system configuration sections [SYSTEM] cuda_devices = "" num_threads = 1 num_gpus = 1 model_dir = ../model
[NETWORK] name = unet_2d activation_function = relu batch_size = 10 decay = 0 reg_type = L2
volume_padding_size = (92,92)
histogram normalisation histogram_ref_file = ../standardisation_models.txt norm_type = percentile cutoff = (0.01, 0.99) normalisation = True whitening = True normalise_foreground_only = True foreground_type = otsu_plus multimod_foreground_type = and
queue_length = 25 window_sampling = uniform
[TRAINING] optimiser = adam sample_per_volume = 32
rotation_angle = (-10.0, 10.0) scaling_percentage = (-10.0, 10.0) random_flipping_axes= 1 lr = 0.000001 loss_type = Dice starting_iter = 0 save_every_n = 30000 max_iter = 30000 max_checkpoints = 100
[INFERENCE] border = 0 inference_iter = -1 save_seg_dir = ../segout output_interp_order = 0 spatial_window_size = (236, 236)
############################ custom configuration sections [SEGMENTATION]
describes how to combine multiple input sources image = T1,T2,T1ce,Flair label = label
weight = T1 output_prob = False num_classes = 4 label_normalisation = True
I got this error : ResourceExhaustedError (see above for traceback): OOM when allocating tensor of shape [] and type float [[{{node UNet2D/TwoConv_4/conv_relu1/conv/w/Initializer/truncated_normal/stddev}} = Const[_class=["loc:@UNet2D/TwoConv_4/conv_relu1/conv/w/Assign"], dtype=DT_FLOAT, value=Tensor<type: float shape: [] values: 0.0167473406>, _device="/job:localhost/replica:0/task:0/device:GPU:0"]()]]
when trying to run 2D-unet on GPU for cells segmentation tasks following the new tutorial in ./demos/unet If I change the mode to run on CPU then everything is fine. Why would this happen?