search

CSBDeep / CSBDeep_website

40 stars 3 forks source link

When passing an IntervalView to CSBDeep, the lower limits of the interval seem not to be taken into account correctly. #10

Open michaelmell opened 5 years ago

michaelmell commented 5 years ago

When passing an IntervalView to CSBDeep, the lower limits of the interval seem not to be taken into account correctly.

This works for a model with input-tensor size 32 x 512:

FinalInterval roiForNetworkProcessing = new FinalInterval(new long[]{37, img.dimension(1) - 512, 0}, new long[]{68, img.dimension(1) - 1, img.dimension(2) - 1});
IntervalView<FloatType> newImg = Views.interval(img, roiForNetworkProcessing);

Dataset dataset = datasetService.create(Views.zeroMin(newImg)); // WHY DO WE NEED ZEROMIN HERE?!

final CommandModule module = commandService.run(
    GenericNetwork.class, false,
    "input", dataset,
    "modelFile", "/path/to//model.zip",
    "normalizeInput", false,
    "blockMultiple", 8,
    "nTiles", 1,
    "showProgressDialog", true).get();
Img<FloatType> tmp = (Img<FloatType>) module.getOutput("output");

But changing the line

Dataset dataset = datasetService.create(Views.zeroMin(newImg)); // WHY DO WE NEED ZEROMIN HERE?!

into

Dataset dataset = datasetService.create(newImg); // WHY DO WE NEED ZEROMIN HERE?!

causes the error/exception below, where the reported network input size is noteworthy:

[INFO] Network input size: [69, 531, 1, 1]

From this, it seems that only the upper limits of the IntervalView are taken into account. Note that the size of the input image is 113 x 531.

This complete exception output:

[INFO] Shape of input tensor: [-1, 512, 32, 1]
[INFO] Shape of output tensor: [-1, 512, 32, 1]
[INFO] Dataset type: 32-bit signed float, converting to FloatType.
[INFO] Dataset dimensions: [32, 512, 6]
[INFO] INPUT NODE:
[INFO] Mapping of tensor input:
[INFO]    datasetAxes:[X, Y, Z]
[INFO]    nodeAxes:[(Z, -1), (Y, 512), (X, 32), (Channel, 1)]
[INFO]    mapping:[2, 1, 0, 3]
[INFO] OUTPUT NODE:
[INFO] Mapping of tensor output:
[INFO]    datasetAxes:[X, Y, Z]
[INFO]    nodeAxes:[(Z, -1), (Y, 512), (X, 32), (Channel, 1)]
[INFO]    mapping:[2, 1, 0, 3]
[INFO] Complete input axes: [X, Y, Z, Channel]
[INFO] Tiling actions: [NO_TILING, NO_TILING, TILE_WITHOUT_PADDING, NO_TILING]
[INFO] Dividing image into 6 tile(s)..
[INFO] Size of single image tile: [32, 512, 1, 1]
[INFO] Final image tiling: [1, 1, 6, 1]
[INFO] Network input size: [69, 531, 1, 1]
[INFO] Processing tile 1..
java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: ConcatOp : Dimensions of inputs should match: shape[0] = [1,132,16,128] vs. shape[1] = [1,132,17,128]
     [[{{node concatenate_2/concat}} = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32, _output_shapes=[[?,128,8,256]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv2d_transpose_2/BiasAdd, conv2d_6/Relu, concatenate_1/concat/axis)]]
    at java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1006)
    at de.csbdresden.csbdeep.network.DefaultModelExecutor.run(DefaultModelExecutor.java:82)
    at de.csbdresden.csbdeep.network.DefaultModelExecutor.run(DefaultModelExecutor.java:43)
    at de.csbdresden.csbdeep.commands.GenericNetwork.tileAndRunNetwork(GenericNetwork.java:559)
    at de.csbdresden.csbdeep.commands.GenericNetwork.tryToTileAndRunNetwork(GenericNetwork.java:537)
    at de.csbdresden.csbdeep.commands.GenericNetwork.mainThread(GenericNetwork.java:449)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
    at java.util.concurrent.FutureTask.run(FutureTask.java)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: ConcatOp : Dimensions of inputs should match: shape[0] = [1,132,16,128] vs. shape[1] = [1,132,17,128]
     [[{{node concatenate_2/concat}} = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32, _output_shapes=[[?,128,8,256]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv2d_transpose_2/BiasAdd, conv2d_6/Relu, concatenate_1/concat/axis)]]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:593)
    at java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1005)
    ... 11 more
Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: ConcatOp : Dimensions of inputs should match: shape[0] = [1,132,16,128] vs. shape[1] = [1,132,17,128]
     [[{{node concatenate_2/concat}} = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32, _output_shapes=[[?,128,8,256]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv2d_transpose_2/BiasAdd, conv2d_6/Relu, concatenate_1/concat/axis)]]
    at java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1431)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: ConcatOp : Dimensions of inputs should match: shape[0] = [1,132,16,128] vs. shape[1] = [1,132,17,128]
     [[{{node concatenate_2/concat}} = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32, _output_shapes=[[?,128,8,256]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv2d_transpose_2/BiasAdd, conv2d_6/Relu, concatenate_1/concat/axis)]]
    at java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.util.concurrent.FutureTask.get(FutureTask.java:192)
    at de.csbdresden.csbdeep.network.model.DefaultNetwork.call(DefaultNetwork.java:86)
    at de.csbdresden.csbdeep.network.model.DefaultNetwork.call(DefaultNetwork.java:23)
    at java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1424)
    ... 4 more
Caused by: java.lang.IllegalArgumentException: ConcatOp : Dimensions of inputs should match: shape[0] = [1,132,16,128] vs. shape[1] = [1,132,17,128]
     [[{{node concatenate_2/concat}} = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32, _output_shapes=[[?,128,8,256]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv2d_transpose_2/BiasAdd, conv2d_6/Relu, concatenate_1/concat/axis)]]
    at org.tensorflow.Session.run(Native Method)
    at org.tensorflow.Session.access$100(Session.java:48)
    at org.tensorflow.Session$Runner.runHelper(Session.java:314)
    at org.tensorflow.Session$Runner.run(Session.java:264)
    at de.csbdresden.csbdeep.network.model.tensorflow.TensorFlowRunner.executeGraph(TensorFlowRunner.java:54)
    at de.csbdresden.csbdeep.network.model.tensorflow.TensorFlowNetwork.execute(TensorFlowNetwork.java:345)
    at de.csbdresden.csbdeep.network.model.DefaultNetwork.lambda$call$0(DefaultNetwork.java:78)
    at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
    at java.util.concurrent.FutureTask.run(FutureTask.java)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: ConcatOp : Dimensions of inputs should match: shape[0] = [1,132,16,128] vs. shape[1] = [1,132,17,128]
     [[{{node concatenate_2/concat}} = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32, _output_shapes=[[?,128,8,256]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv2d_transpose_2/BiasAdd, conv2d_6/Relu, concatenate_1/concat/axis)]]
    at java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1006)
    at de.csbdresden.csbdeep.network.DefaultModelExecutor.run(DefaultModelExecutor.java:82)
    at de.csbdresden.csbdeep.network.DefaultModelExecutor.run(DefaultModelExecutor.java:43)
    at de.csbdresden.csbdeep.commands.GenericNetwork.tileAndRunNetwork(GenericNetwork.java:559)
    at de.csbdresden.csbdeep.commands.GenericNetwork.tryToTileAndRunNetwork(GenericNetwork.java:537)
    at de.csbdresden.csbdeep.commands.GenericNetwork.mainThread(GenericNetwork.java:449)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
    at java.util.concurrent.FutureTask.run(FutureTask.java)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: ConcatOp : Dimensions of inputs should match: shape[0] = [1,132,16,128] vs. shape[1] = [1,132,17,128]
     [[{{node concatenate_2/concat}} = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32, _output_shapes=[[?,128,8,256]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv2d_transpose_2/BiasAdd, conv2d_6/Relu, concatenate_1/concat/axis)]]
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at java.util.concurrent.ForkJoinTask.getThrowableException(ForkJoinTask.java:593)
    at java.util.concurrent.ForkJoinTask.get(ForkJoinTask.java:1005)
    ... 11 more
Caused by: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: ConcatOp : Dimensions of inputs should match: shape[0] = [1,132,16,128] vs. shape[1] = [1,132,17,128]
     [[{{node concatenate_2/concat}} = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32, _output_shapes=[[?,128,8,256]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv2d_transpose_2/BiasAdd, conv2d_6/Relu, concatenate_1/concat/axis)]]
    at java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1431)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException: ConcatOp : Dimensions of inputs should match: shape[0] = [1,132,16,128] vs. shape[1] = [1,132,17,128]
     [[{{node concatenate_2/concat}} = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32, _output_shapes=[[?,128,8,256]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv2d_transpose_2/BiasAdd, conv2d_6/Relu, concatenate_1/concat/axis)]]
    at java.util.concurrent.FutureTask.report(FutureTask.java:122)
    at java.util.concurrent.FutureTask.get(FutureTask.java:192)
    at de.csbdresden.csbdeep.network.model.DefaultNetwork.call(DefaultNetwork.java:86)
    at de.csbdresden.csbdeep.network.model.DefaultNetwork.call(DefaultNetwork.java:23)
    at java.util.concurrent.ForkJoinTask$AdaptedCallable.exec(ForkJoinTask.java:1424)
    ... 4 more
Caused by: java.lang.IllegalArgumentException: ConcatOp : Dimensions of inputs should match: shape[0] = [1,132,16,128] vs. shape[1] = [1,132,17,128]
     [[{{node concatenate_2/concat}} = ConcatV2[N=2, T=DT_FLOAT, Tidx=DT_INT32, _output_shapes=[[?,128,8,256]], _device="/job:localhost/replica:0/task:0/device:CPU:0"](conv2d_transpose_2/BiasAdd, conv2d_6/Relu, concatenate_1/concat/axis)]]
    at org.tensorflow.Session.run(Native Method)
    at org.tensorflow.Session.access$100(Session.java:48)
    at org.tensorflow.Session$Runner.runHelper(Session.java:314)
    at org.tensorflow.Session$Runner.run(Session.java:264)
    at de.csbdresden.csbdeep.network.model.tensorflow.TensorFlowRunner.executeGraph(TensorFlowRunner.java:54)
    at de.csbdresden.csbdeep.network.model.tensorflow.TensorFlowNetwork.execute(TensorFlowNetwork.java:345)
    at de.csbdresden.csbdeep.network.model.DefaultNetwork.lambda$call$0(DefaultNetwork.java:78)
    at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
    at java.util.concurrent.FutureTask.run(FutureTask.java)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)
[INFO] Plugin exit (took 1273 milliseconds)
java.lang.NullPointerException
    at net.imglib2.AbstractInterval.<init>(AbstractInterval.java:72)
    at net.imglib2.FinalInterval.<init>(FinalInterval.java:54)
    at net.imglib2.loops.LoopBuilder.<init>(LoopBuilder.java:88)
    at net.imglib2.loops.LoopBuilder.setImages(LoopBuilder.java:107)
    at com.jug.MoMA.runNetwork(MoMA.java:1885)
    at com.jug.MoMA.generateAllSimpleSegmentationHypotheses(MoMA.java:1802)
    at com.jug.MoMA.restartFromGLSegmentation(MoMA.java:2048)
    at com.jug.MoMA.processDataFromFolder(MoMA.java:1425)
    at com.jug.MoMA.main(MoMA.java:725)
Disconnected from the target VM, address: '127.0.0.1:34549', transport: 'socket'

Process finished with exit code 11