support for channels_first

[Neo-X]

I seem to be getting an error when I am using "channels_first" in keras. I don't have the issue when using "channels_last"

The building of the network goes okay but when I pass in an input that is channels_first I get the following error.

Caused by op 'coordinate_channel2d_4/concat', defined at:
  File "trainModel.py", line 1395, in <module>
    simData = trainModelParallel((sys.argv[1], settings))
  File "trainModel.py", line 261, in trainModelParallel
    sw.start()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start
    self._popen = self._Popen(self)
  File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/usr/lib/python3.6/multiprocessing/context.py", line 277, in _Popen
    return Popen(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 20, in __init__
    self._launch(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 74, in _launch
    code = process_obj._bootstrap()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
    self.run()
  File "/home/gberseth/playground/RL-Framework/simulation/SimWorker.py", line 148, in run
    self._model.setPolicy(self.createNewModel())
  File "/home/gberseth/playground/RL-Framework/simulation/SimWorker.py", line 53, in createNewModel
    np.array(self._settings['reward_bounds']), self._settings)
  File "/home/gberseth/playground/RL-Framework/util/SimulationUtil.py", line 281, in createRLAgent
    networkModel = createNetworkModel(settings["model_type"], state_bounds, action_bounds, reward_bounds, settings, print_info=print_info)
  File "/home/gberseth/playground/RL-Framework/util/SimulationUtil.py", line 259, in createNetworkModel
    print_info=print_info)
  File "/home/gberseth/playground/RL-Framework/model/DeepNNKerasAdaptive.py", line 377, in __init__
    network = CoordinateChannel2D()(network)
  File "/home/gberseth/.local/lib/python3.6/site-packages/keras/engine/topology.py", line 619, in __call__
    output = self.call(inputs, **kwargs)
  File "/home/gberseth/playground/RL-Framework/util/coordconv.py", line 124, in call
    outputs = K.concatenate([inputs, xx_channels, yy_channels], axis=-1)
  File "/home/gberseth/.local/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 1880, in concatenate
    return tf.concat([to_dense(x) for x in tensors], axis)
  File "/home/gberseth/.local/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1099, in concat
    return gen_array_ops._concat_v2(values=values, axis=axis, name=name)
  File "/home/gberseth/.local/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 706, in _concat_v2
    "ConcatV2", values=values, axis=axis, name=name)
  File "/home/gberseth/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/gberseth/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2956, in create_op
    op_def=op_def)
  File "/home/gberseth/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1470, in __init__
    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access

InvalidArgumentError (see above for traceback): ConcatOp : Dimensions of inputs should match: shape[0] = [1,32,32,3] vs. shape[1] = [1,3,32,1]
     [[Node: coordinate_channel2d_4/concat = ConcatV2[N=3, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/device:CPU:0"](coordinate_channel2d_4/transpose, coordinate_channel2d_4/sub_1, coordinate_channel2d_4/sub_3, reshape_2/Reshape/shape/1)]]

My original input shape is (3,32,32) with 3 channels.

[dongrongliang]

I got the same error:

Traceback (most recent call last):
  File "C:\Users\domer\AppData\Local\conda\conda\envs\tensorflow-gpu\lib\site-packages\tensorflow\python\client\session.py", line 1323, in _do_call
    return fn(*args)
  File "C:\Users\domer\AppData\Local\conda\conda\envs\tensorflow-gpu\lib\site-packages\tensorflow\python\client\session.py", line 1302, in _run_fn
    status, run_metadata)
  File "C:\Users\domer\AppData\Local\conda\conda\envs\tensorflow-gpu\lib\site-packages\tensorflow\python\framework\errors_impl.py", line 473, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.InvalidArgumentError: ConcatOp : Dimensions of inputs should match: shape[0] = [1,288,432,3] vs. shape[1] = [1,3,288,1]
  [[Node: coordinate_channel2d_1/concat = ConcatV2[N=3, T=DT_FLOAT, Tidx=DT_INT32, _device="/job:localhost/replica:0/task:0/device:GPU:0"](coordinate_channel2d_1/transpose, coordinate_channel2d_1/sub_1, coordinate_channel2d_1/sub_3, coordinate_channel2d_1/concat/axis)]]
  [[Node: loss/mul/_135 = _Recv[client_terminated=false, recv_device="/job:localhost/replica:0/task:0/device:CPU:0", send_device="/job:localhost/replica:0/task:0/device:GPU:0", send_device_incarnation=1, tensor_name="edge_688_loss/mul", tensor_type=DT_FLOAT, _device="/job:localhost/replica:0/task:0/device:CPU:0"]()]]

[dongrongliang]

I found a solution for this issue, just add one line code after input permutation;

if self.data_format == 'channels_first':
    inputs = K.permute_dimensions(inputs, [0, 2, 3, 1])
    input_shape = K.shape(inputs)

[Neo-X]

This appears to fix part of the issue! My code will run fine how it was but if I add pooling layers between the convolutional layers I get the following error.

   data_format=data_format_)(networkAct)
  File "/home/gberseth/.local/lib/python3.6/site-packages/keras/engine/topology.py", line 619, in __call__
    output = self.call(inputs, **kwargs)
  File "/home/gberseth/.local/lib/python3.6/site-packages/keras/layers/pooling.py", line 158, in call
    data_format=self.data_format)
  File "/home/gberseth/.local/lib/python3.6/site-packages/keras/layers/pooling.py", line 221, in _pooling_function
    pool_mode='max')
  File "/home/gberseth/.local/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 3658, in pool2d
    data_format=tf_data_format)
  File "/home/gberseth/.local/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 1958, in max_pool
    name=name)
  File "/home/gberseth/.local/lib/python3.6/site-packages/tensorflow/python/ops/gen_nn_ops.py", line 2806, in _max_pool
    data_format=data_format, name=name)
  File "/home/gberseth/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/gberseth/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2958, in create_op
    set_shapes_for_outputs(ret)
  File "/home/gberseth/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2209, in set_shapes_for_outputs
    shapes = shape_func(op)
  File "/home/gberseth/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 2159, in call_with_requiring
    return call_cpp_shape_fn(op, require_shape_fn=True)
  File "/home/gberseth/.local/lib/python3.6/site-packages/tensorflow/python/framework/common_shapes.py", line 627, in call_cpp_shape_fn
    require_shape_fn)
  File "/home/gberseth/.local/lib/python3.6/site-packages/tensorflow/python/framework/common_shapes.py", line 691, in _call_cpp_shape_fn_impl
    raise ValueError(err.message)
ValueError: Negative dimension size caused by subtracting 2 from 1 for 'max_pooling2d_3/MaxPool' (op: 'MaxPool') with input shapes: [?,1,1,16].

[titu1994]

@Neo-X Your image is too small to allow another pooling layer. The fix suggested above is correct and I'll merge it in a short while.

[titu1994]

Closed via https://github.com/titu1994/keras-coordconv/commit/c045e3f1ff7dabd4060f515e4b900263eddf1723