search

hellochick / PWCNet-tf2

TensorFlow-based implementation of "PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volume".
MIT License
27 stars 8 forks source link

on kitti dataset #1

Open thisiskiru opened 4 years ago

thisiskiru commented 4 years ago

error in inference.py

thisiskiru commented 4 years ago

the following error raised while running inference.py main() File "inference.py", line 76, in main flo_pred = inference(image_pairs, model=pwcnet) File "C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\eager\def_function.py", line 580, in call result = self._call(*args, kwds) File "C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\eager\def_function.py", line 627, in _call self._initialize(args, kwds, add_initializers_to=initializers) File "C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\eager\def_function.py", line 506, in _initialize *args, *kwds)) File "C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\eager\function.py", line 2446, in _get_concrete_function_internal_garbage_collected graphfunction, , _ = self._maybe_define_function(args, kwargs) File "C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\eager\function.py", line 2777, in _maybe_define_function graph_function = self._create_graph_function(args, kwargs) File "C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\eager\function.py", line 2667, in _create_graph_function capture_by_value=self._capture_by_value), File "C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\framework\func_graph.py", line 981, in func_graph_from_py_func func_outputs = python_func(func_args, func_kwargs) File "C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\eager\def_function.py", line 441, in wrapped_fn return weak_wrapped_fn().wrapped(*args, **kwds) File "C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\framework\func_graph.py", line 968, in wrapper raise e.ag_error_metadata.to_exception(e) tensorflow.python.framework.errors_impl.OutOfRangeError: in user code:

inference.py:53 inference  *
    flo_pred = model(image_pairs, is_training=False)
E:\LiteFlowNet2-TF2-master\PWCNet-tf2-master\PWCNet-tf2-master\PWCDCNet.py:175 call  *
    c11 = self.conv1b(self.conv1aa(self.conv1a(im1)))
E:\LiteFlowNet2-TF2-master\PWCNet-tf2-master\PWCNet-tf2-master\PWCDCNet.py:23 call  *
    x = self.conv_out(inputs)
C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\keras\engine\base_layer.py:897 __call__  **
    self._maybe_build(inputs)
C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\keras\engine\base_layer.py:2416 _maybe_build
    self.build(input_shapes)  # pylint:disable=not-callable
C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\keras\layers\convolutional.py:163 build
    dtype=self.dtype)
C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\keras\engine\base_layer.py:577 add_weight
    caching_device=caching_device)
C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\training\tracking\base.py:724 _add_variable_with_custom_getter
    name=name, shape=shape)
C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\training\tracking\base.py:791 _preload_simple_restoration
    checkpoint_position=checkpoint_position, shape=shape)
C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\training\tracking\base.py:71 __init__
    self.wrapped_value = checkpoint_position.value_tensors()[VARIABLE_VALUE_KEY]
C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\training\tracking\base.py:304 value_tensors
    name="%s_checkpoint_read" % (serialized_tensor.name,))
C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\ops\gen_io_ops.py:1492 restore_v2
    ctx=_ctx)
C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\ops\gen_io_ops.py:1532 restore_v2_eager_fallback
    attrs=_attrs, ctx=ctx, name=name)
C:\Users\CVR 2019 2020\Anaconda3\envs\tf2\lib\site-packages\tensorflow\python\eager\execute.py:60 quick_execute
    inputs, attrs, num_outputs)

OutOfRangeError: Read fewer bytes than requested [Op:RestoreV2]
mjoannou commented 4 years ago

Seconded, this error occurs using the sample images provided in the repository. Specifically the error arises during the 3rd line of PWCDCNet.call(), when you attempt to pass im1 to the first convolutional layer self.conv1a(im1)

thisiskiru commented 4 years ago

Yes I see that but it even rise a error using other (not sample images provided in the repository) images too...

hellochick commented 4 years ago

Hi,

I'm sorry for your inconvenience, this repository works only with FlyingChairs and FlyingThings now.

If you are trying to train on Sintel or Kitti, there are several things to be noticed as below:

  1. I have tried this code on Sintel, and the test results are not desirable, which means it cannot reproduce the test results in the original paper.

  2. As for issue (1), I have tried several ways to improve it (not shown in this repo), but it still cannot reach the performance: (a) Re-write the data augmentation part (you can refer to this paper to see the importance of data augmentation to optical flow). I recommend people can refer to the augmentation part of hd3 and MaskFlowNet.

Issues that raised and related to the (a):

  1. It still not clear whether to rescale the flow ground truth or not.