search

NifTK / NiftyNet

[unmaintained] An open-source convolutional neural networks platform for research in medical image analysis and image-guided therapy
http://niftynet.io
Apache License 2.0
1.37k stars 404 forks source link

Niftynet 0.3.0 Unet error: Out of bounds slicing on dimension 1 of length 64: #125

Closed JunMa11 closed 6 years ago

JunMa11 commented 6 years ago

Dear niftynet organizers, Firstly, thanks for the excellent work. There is something wrong with stack overflow, so I post the error here.

Stack Overflow requires external JavaScript from another domain, which is blocked or failed to load.

I try Unet for 3D segmentation. Following is the config file.

[ImgData]
path_to_search = F:\\ROI_Data\\
filename_contains = ROI_Data_
spatial_window_size = (64,64,64)
axcodes=(A, R, S)
interp_order = 1

[ImgLabel]
path_to_search = F:\\ROI_Label\\
filename_contains = ROI_label_
spatial_window_size = (64,64,64)
axcodes=(A, R, S)
interp_order = 0

[NETWORK]
name = unet
activation_function = prelu
batch_size = 2
decay = 0.00001
reg_type = L2

# volume level preprocessing
volume_padding_size = 0

normalisation = False
whitening = False
normalise_foreground_only=False

queue_length = 6
window_sampling = resize

[TRAINING]
sample_per_volume = 1
rotation_angle = (-10.0, 10.0)
scaling_percentage = (-10.0, 10.0)
random_flipping_axes= 0,1,2

lr = 0.0001
loss_type = Dice
starting_iter = 0
save_every_n = 4000
max_iter = 40000
max_checkpoints = 20

validation_every_n = 25
exclude_fraction_for_validation = 0.1
exclude_fraction_for_inference = 0.2

[SEGMENTATION]
image = ImgData
label = ImgLabel
output_prob = False
num_classes = 2
label_normalisation = False

min_numb_labels = 2
compulsory_labels=0, 1
min_sampling_ratio = 0.0001

In my mind, unet is ok for 3D image segmentation. However, when I update the niftynet version to 0.3.0, the following error message occurs. 

2018-06-15 23:13:48.829294: W T:\src\github\tensorflow\tensorflow\core\kernels\queue_base.cc:277]

 _0_Sampler/shuffled_queue_1: Skipping cancelled enqueue attempt with queue not closed
2018-06-15 23:13:48.830294: W T:\src\github\tensorflow\tensorflow\core\kernels\queue_base.cc:277] 

_0_Sampler/shuffled_queue_1: Skipping cancelled enqueue attempt with queue not closed
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 686, in _call_cpp_shape_fn_impl
    input_tensors_as_shapes, status)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 516, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: Out of bounds slicing on dimension 1  
of length 64: sizes vector cannot be < -1, but was -24 for 'worker_0/UNet/crop-88/Slice' (op: 'Slice' 
) with input shapes: [2,64,64,64,2], [5], [5] and with computed input tensors: 
input[2] = <-1 -24 -24 -24 -1> and with input tensors computed as partial shapes: input[1] = [0,44,44,44,0].
During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "net_segment.py", line 8, in <module>
    sys.exit(main())
  File "F:\LiverSeg0519\NiftyNet\niftynet\__init__.py", line 139, in main
    app_driver.run_application()

  File "F:\LiverSeg0519\NiftyNet\niftynet\engine\application_driver.py", line 255, in run_application
    self.graph = self._create_graph(self.graph)
  File "F:\LiverSeg0519\NiftyNet\niftynet\engine\application_driver.py", line 331, in _create_graph
    self.gradients_collector)
  File "F:\LiverSeg0519\NiftyNet\niftynet\application\segmentation_application.py", line 310, in connect_data_and_network
    net_out = self.net(image, is_training=self.is_training)
  File "F:\LiverSeg0519\NiftyNet\niftynet\layer\base_layer.py", line 34, in __call__
    return self._op(*args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\template.py", line 455, in __call__
    result = self._call_func(args, kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\template.py", line 406, in _call_func
    result = self._func(*args, **kwargs)
  File "F:\LiverSeg0519\NiftyNet\niftynet\network\unet.py", line 122, in layer_op
    output_tensor = crop_layer(output_tensor)
  File "F:\LiverSeg0519\NiftyNet\niftynet\layer\base_layer.py", line 34, in __call__
    return self._op(*args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\template.py", line 455, in __call__
    result = self._call_func(args, kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\template.py", line 406, in _call_func
    result = self._func(*args, **kwargs)
  File "F:\LiverSeg0519\NiftyNet\niftynet\layer\crop.py", line 26, in layer_op
    output_tensor = tf.slice(inputs, offsets, out_shape)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\array_ops.py", line 647, in slice
    return gen_array_ops._slice(input_, begin, size, name=name)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 8240, in _slice
    "Slice", input=input, begin=begin, size=size, name=name)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 3292, in create_op
    compute_device=compute_device)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 3332, in _create_op_helper
    set_shapes_for_outputs(op)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 2496, in set_shapes_for_outputs
    return _set_shapes_for_outputs(op)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 2469, in _set_shapes_for_outputs
    shapes = shape_func(op)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 2399, in call_with_requiring
    return call_cpp_shape_fn(op, require_shape_fn=True)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 627, in call_cpp_shape_fn
    require_shape_fn)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 691, in _call_cpp_shape_fn_impl
    raise ValueError(err.message)
ValueError: Out of bounds slicing on dimension 1 of length 64: sizes vector cannot be 
< -1, but was -24 for 'worker_0/UNet/crop-88/Slice' (op: 'Slice') with 
input shapes: [2,64,64,64,2], [5], [5] and
with computed input tensors: input[2] = <-1 -24 -24 -24 -1> and 
with input tensors computed as partial shapes: input[1] = [0,44,44,44,0].
originally defined at:
  File "F:\LiverSeg0519\NiftyNet\niftynet\network\unet.py", line 121, in layer_op
    crop_layer = CropLayer(border=44, name='crop-88')
  File "F:\LiverSeg0519\NiftyNet\niftynet\layer\crop.py", line 18, in __init__
    super(CropLayer, self).__init__(name=name)
  File "F:\LiverSeg0519\NiftyNet\niftynet\layer\base_layer.py", line 26, in __init__
    self._op = tf.make_template(name, self.layer_op, create_scope_now_=True)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\template.py", line 153, in make_template
    **kwargs)

originally defined at:
  File "F:\LiverSeg0519\NiftyNet\niftynet\engine\application_driver.py", line 317, in _create_graph
    self.app.initialise_network()
  File "F:\LiverSeg0519\NiftyNet\niftynet\application\segmentation_application.py", line 278, in initialise_network
    acti_func=self.net_param.activation_function)
  File "F:\LiverSeg0519\NiftyNet\niftynet\network\unet.py", line 29, in __init__
    super(UNet3D, self).__init__(name=name)
  File "F:\LiverSeg0519\NiftyNet\niftynet\layer\base_layer.py", line 58, in __init__
    super(TrainableLayer, self).__init__(name=name)
  File "F:\LiverSeg0519\NiftyNet\niftynet\layer\base_layer.py", line 26, in __init__
    self._op = tf.make_template(name, self.layer_op, create_scope_now_=True)
  File "F:\LiverSeg0519\NiftyNet\niftynet\engine\application_driver.py", line 255, in run_application
    self.graph = self._create_graph(self.graph)
  File "F:\LiverSeg0519\NiftyNet\niftynet\engine\application_driver.py", line 331, in _create_graph
    self.gradients_collector)
  File "F:\LiverSeg0519\NiftyNet\niftynet\application\segmentation_application.py", line 310, in connect_data_and_network
    net_out = self.net(image, is_training=self.is_training)
  File "F:\LiverSeg0519\NiftyNet\niftynet\layer\base_layer.py", line 34, in __call__
    return self._op(*args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\template.py", line 455, in __call__
    result = self._call_func(args, kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\template.py", line 406, in _call_func
    result = self._func(*args, **kwargs)
  File "F:\LiverSeg0519\NiftyNet\niftynet\network\unet.py", line 122, in layer_op
    output_tensor = crop_layer(output_tensor)
  File "F:\LiverSeg0519\NiftyNet\niftynet\layer\base_layer.py", line 34, in __call__
    return self._op(*args, **kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\template.py", line 455, in __call__
    result = self._call_func(args, kwargs)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\template.py", line 406, in _call_func
    result = self._func(*args, **kwargs)
  File "F:\LiverSeg0519\NiftyNet\niftynet\layer\crop.py", line 26, in layer_op
    output_tensor = tf.slice(inputs, offsets, out_shape)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\array_ops.py", line 647, in slice
    return gen_array_ops._slice(input_, begin, size, name=name)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\gen_array_ops.py", line 8240, in _slice
    "Slice", input=input, begin=begin, size=size, name=name)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 3292, in create_op
    compute_device=compute_device)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 3332, in _create_op_helper
    set_shapes_for_outputs(op)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 2496, in set_shapes_for_outputs
    return _set_shapes_for_outputs(op)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 2469, in _set_shapes_for_outputs
    shapes = shape_func(op)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\ops.py", line 2399, in call_with_requiring
    return call_cpp_shape_fn(op, require_shape_fn=True)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 627, in call_cpp_shape_fn
    require_shape_fn)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\common_shapes.py", line 691, in _call_cpp_shape_fn_impl
    raise ValueError(err.message)
ValueError: Out of bounds slicing on dimension 1 of length 64: sizes vector 
cannot be < -1, but was -24 for 'worker_0/UNet/crop-88/Slice' (op: 'Slice') with 
input shapes: [2,64,64,64,2], [5], [5] and
with computed input tensors: input[2] = <-1 -24 -24 -24 -1> and with 
input tensors computed as partial shapes: input[1] = [0,44,44,44,0].

originally defined at:
  File "F:\LiverSeg0519\NiftyNet\niftynet\network\unet.py", line 121, in layer_op
    crop_layer = CropLayer(border=44, name='crop-88')
  File "F:\LiverSeg0519\NiftyNet\niftynet\layer\crop.py", line 18, in __init__
    super(CropLayer, self).__init__(name=name)
  File "F:\LiverSeg0519\NiftyNet\niftynet\layer\base_layer.py", line 26, in __init__
    self._op = tf.make_template(name, self.layer_op, create_scope_now_=True)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\ops\template.py", line 153, in make_template
    **kwargs)

originally defined at:
  File "F:\LiverSeg0519\NiftyNet\niftynet\engine\application_driver.py", line 317, in _create_graph
    self.app.initialise_network()
  File "F:\LiverSeg0519\NiftyNet\niftynet\application\segmentation_application.py", line 278, in initialise_network
    acti_func=self.net_param.activation_function)
  File "F:\LiverSeg0519\NiftyNet\niftynet\network\unet.py", line 29, in __init__
    super(UNet3D, self).__init__(name=name)
  File "F:\LiverSeg0519\NiftyNet\niftynet\layer\base_layer.py", line 58, in __init__
    super(TrainableLayer, self).__init__(name=name)
  File "F:\LiverSeg0519\NiftyNet\niftynet\layer\base_layer.py", line 26, in __init__
    self._op = tf.make_template(name, self.layer_op, create_scope_now_=True)

Would it be possible for you to give me some advice to solve this problem. Looking forward to your reply.

Best regards, Edward

wyli commented 6 years ago

duplicates of https://github.com/NifTK/NiftyNet/issues/17 and https://github.com/NifTK/NiftyNet/issues/44 ?

JunMa11 commented 6 years ago

@wyli Thanks for your quick reply. I remove the cropping layer in the unet. But I obtain the following error:

Executor failed to create kernel. Invalid argument: AttrValue must not have reference type value of float_ref for attr 'tensor_type'

INFO:niftynet: starting from iter 0
2018-06-16 13:03:14.888422: E T:\src\github\tensorflow\tensorflow\core\framework\op_segment.cc:53] Create kernel failed: Invalid argument: AttrValue must not have reference type value of float_ref
         for attr 'tensor_type'
        ; NodeDef: UNet/R3/256_bn_prelu/bn_/beta/_7 = _Recv[_start_time=0, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/ta
sk:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_290_UNet/R3/256_bn_prelu/bn_/beta", tensor_type=DT_FLOAT_REF, _device="/job:localhost/replica:0/task:0/device:CPU:0"](^UNet/R3/256_bn_p
relu/bn_/beta/Initializer/Const); Op<name=_Recv; signature= -> tensor:tensor_type; attr=tensor_type:type; attr=tensor_name:string; attr=send_device:string; attr=send_device_incarnation:int; attr=recv_
device:string; attr=client_terminated:bool,default=false; is_stateful=true>
2018-06-16 13:03:14.892422: E T:\src\github\tensorflow\tensorflow\core\common_runtime\executor.cc:644] Executor failed to create kernel. Invalid argument: AttrValue must not have reference type value
of float_ref
         for attr 'tensor_type'
        ; NodeDef: UNet/R3/256_bn_prelu/bn_/beta/_7 = _Recv[_start_time=0, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/ta
sk:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_290_UNet/R3/256_bn_prelu/bn_/beta", tensor_type=DT_FLOAT_REF, _device="/job:localhost/replica:0/task:0/device:CPU:0"](^UNet/R3/256_bn_p
relu/bn_/beta/Initializer/Const); Op<name=_Recv; signature= -> tensor:tensor_type; attr=tensor_type:type; attr=tensor_name:string; attr=send_device:string; attr=send_device_incarnation:int; attr=recv_
device:string; attr=client_terminated:bool,default=false; is_stateful=true>
         [[Node: UNet/R3/256_bn_prelu/bn_/beta/_7 = _Recv[_start_time=0, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task
:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_290_UNet/R3/256_bn_prelu/bn_/beta", tensor_type=DT_FLOAT_REF, _device="/job:localhost/replica:0/task:0/device:CPU:0"](^UNet/R3/256_bn_pre
lu/bn_/beta/Initializer/Const)]]
2018-06-16 13:03:14.908423: W T:\src\github\tensorflow\tensorflow\core\kernels\queue_base.cc:277] _0_Sampler/shuffled_queue_1: Skipping cancelled enqueue attempt with queue not closed
2018-06-16 13:03:14.909423: W T:\src\github\tensorflow\tensorflow\core\kernels\queue_base.cc:277] _0_Sampler/shuffled_queue_1: Skipping cancelled enqueue attempt with queue not closed
Traceback (most recent call last):
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 1327, in _do_call
    return fn(*args)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 1312, in _run_fn
    options, feed_dict, fetch_list, target_list, run_metadata)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 1420, in _call_tf_sessionrun
    status, run_metadata)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 516, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: AttrValue must not have reference type value of float_ref
         for attr 'tensor_type'
        ; NodeDef: UNet/R3/256_bn_prelu/bn_/beta/_7 = _Recv[_start_time=0, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/ta
sk:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_290_UNet/R3/256_bn_prelu/bn_/beta", tensor_type=DT_FLOAT_REF, _device="/job:localhost/replica:0/task:0/device:CPU:0"](^UNet/R3/256_bn_p
relu/bn_/beta/Initializer/Const); Op<name=_Recv; signature= -> tensor:tensor_type; attr=tensor_type:type; attr=tensor_name:string; attr=send_device:string; attr=send_device_incarnation:int; attr=recv_
device:string; attr=client_terminated:bool,default=false; is_stateful=true>
         [[Node: UNet/R3/256_bn_prelu/bn_/beta/_7 = _Recv[_start_time=0, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task
:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_290_UNet/R3/256_bn_prelu/bn_/beta", tensor_type=DT_FLOAT_REF, _device="/job:localhost/replica:0/task:0/device:CPU:0"](^UNet/R3/256_bn_pre
lu/bn_/beta/Initializer/Const)]]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "net_segment.py", line 8, in <module>
    sys.exit(main())
  File "F:\LiverSeg0519\NiftyNet\niftynet\__init__.py", line 139, in main
    app_driver.run_application()
  File "F:\LiverSeg0519\NiftyNet\niftynet\engine\application_driver.py", line 261, in run_application
    self._rand_init_or_restore_vars(session)
  File "F:\LiverSeg0519\NiftyNet\niftynet\engine\application_driver.py", line 369, in _rand_init_or_restore_vars
    sess.run(self._init_op)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 905, in run
    run_metadata_ptr)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 1140, in _run
    feed_dict_tensor, options, run_metadata)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 1321, in _do_run
    run_metadata)
  File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\python\client\session.py", line 1340, in _do_call
    raise type(e)(node_def, op, message)
tensorflow.python.framework.errors_impl.InvalidArgumentError: AttrValue must not have reference type value of float_ref
         for attr 'tensor_type'
        ; NodeDef: UNet/R3/256_bn_prelu/bn_/beta/_7 = _Recv[_start_time=0, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/ta
sk:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_290_UNet/R3/256_bn_prelu/bn_/beta", tensor_type=DT_FLOAT_REF, _device="/job:localhost/replica:0/task:0/device:CPU:0"](^UNet/R3/256_bn_p
relu/bn_/beta/Initializer/Const); Op<name=_Recv; signature= -> tensor:tensor_type; attr=tensor_type:type; attr=tensor_name:string; attr=send_device:string; attr=send_device_incarnation:int; attr=recv_
device:string; attr=client_terminated:bool,default=false; is_stateful=true>
         [[Node: UNet/R3/256_bn_prelu/bn_/beta/_7 = _Recv[_start_time=0, client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task
:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_290_UNet/R3/256_bn_prelu/bn_/beta", tensor_type=DT_FLOAT_REF, _device="/job:localhost/replica:0/task:0/device:CPU:0"](^UNet/R3/256_bn_pre
lu/bn_/beta/Initializer/Const)]]
JunMa11 commented 6 years ago

An excellent explanation of the shape in Unet Url