search

hengchuan / RDN-TensorFlow

A TensorFlow implementation of CVPR 2018 paper "Residual Dense Network for Image Super-Resolution".
MIT License
143 stars 47 forks source link

Phase shift concatenation fails with --scale 1 #11

Open rzumer opened 5 years ago

rzumer commented 5 years ago

I am trying to adapt the network for other applications than super-resolution, using --scale 1. However, in model._phase_shift(), I get the following error, on the concatenation:

  File "main.py", line 53, in <module>
    tf.app.run()
  File "/home/rzumer/dev/tensorflow/lib/python3.6/site-packages/tensorflow/python/platform/app.py", line 125, in run
    _sys.exit(main(argv))
  File "main.py", line 45, in main
    rdn.train(FLAGS)
  File "/home/rzumer/dev/RDN-TensorFlow/model.py", line 278, in train
    self.build_model(images_shape, labels_shape)
  File "/home/rzumer/dev/RDN-TensorFlow/model.py", line 190, in build_model
    self.pred = self.model()
  File "/home/rzumer/dev/RDN-TensorFlow/model.py", line 233, in model
    FU = self.UPN(FDF)
  File "/home/rzumer/dev/RDN-TensorFlow/model.py", line 202, in UPN
    x = self.PS(x, self.scale)
  File "/home/rzumer/dev/RDN-TensorFlow/model.py", line 265, in PS
    X = tf.concat([self._phase_shift(x, r) for x in Xc], 3) # Do the concat RGB
  File "/home/rzumer/dev/RDN-TensorFlow/model.py", line 265, in <listcomp>
    X = tf.concat([self._phase_shift(x, r) for x in Xc], 3) # Do the concat RGB
  File "/home/rzumer/dev/RDN-TensorFlow/model.py", line 246, in _phase_shift
    X = tf.concat([tf.squeeze(x) for x in X], 2)  # bsize, b, a*r, r
  File "/home/rzumer/dev/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/array_ops.py", line 1124, in concat
    return gen_array_ops.concat_v2(values=values, axis=axis, name=name)
  File "/home/rzumer/dev/tensorflow/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 1033, in concat_v2
    "ConcatV2", values=values, axis=axis, name=name)
  File "/home/rzumer/dev/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py", line 787, in _apply_op_helper
    op_def=op_def)
  File "/home/rzumer/dev/tensorflow/lib/python3.6/site-packages/tensorflow/python/util/deprecation.py", line 488, in new_func
    return func(*args, **kwargs)
  File "/home/rzumer/dev/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 3274, in create_op
    op_def=op_def)
  File "/home/rzumer/dev/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1792, in __init__
    control_input_ops)
  File "/home/rzumer/dev/tensorflow/lib/python3.6/site-packages/tensorflow/python/framework/ops.py", line 1631, in _create_c_op
    raise ValueError(str(e))
ValueError: Shape must be at least rank 3 but is rank 2 for 'concat_16' (op: 'ConcatV2') with input shapes: [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [64,32], [] and with computed input tensors: input[32] = <2>.

From my research it seems to be due to attempting to concatenate over a non-existent dimension. However, I am not familiar enough with TensorFlow, RDN, or your implementation to debug this and could not find a reference to phase shifting in the original paper and implementation of RDN, so if --scale 1 is a special case that must be handled separately, please advise on how to make the necessary changes.

I confirmed that --scale 3 works correctly in my environment and restarted from the preprocessing phase to ensure that my data is correct.

hengchuan commented 5 years ago

@rzumer The phase shifting function is a third implement of the pixel shuffle layer in ESPCN. Theoretically it does not change the tensor if scale=1. You could try commenting out the x = self.PS(x, self.scale).