open-mmlab / mmflow

OpenMMLab optical flow toolbox and benchmark
https://mmflow.readthedocs.io/en/latest/
Apache License 2.0
965 stars 116 forks source link

Can't work with np.ndarray input #297

Open JLMarta opened 6 months ago

JLMarta commented 6 months ago

Thanks for your good work and I've checked the following and did not found related one.

Checklist

  1. I have searched related issues but cannot get the expected help.
  2. I have read the FAQ documentation but cannot get the expected help.
  3. The bug has not been fixed in the latest version.

Describe the bug I run the demo.py. The img here is np.ndarray, I've tried both np.unit8 or float type. It would report error below in the trace session. However, this error does not occur when I save the img array above into png and then use png string as input for result = inference_model(model, img1, img2), then this does not happen.

Reproduction

  1. What command or script did you run?
`config_file = r'~\pwcnet_ft_4x1_300k_sintel_final_384x768.py'
checkpoint_file = r'~\pwcnet_ft_4x1_300k_sintel_final_384x768.pth'

# build the model from a config file and a checkpoint file
model = init_model(config_file, checkpoint_file, device='cuda:0')

# test image pair, and save the results
img1=nif[:,:,4,0][40:90,60:110].astype(np.uint8)
img2=nif[:,:,4,1][40:90,60:110].astype(np.uint8)
result = inference_model(model, img1, img2)
  1. Did you make any modifications on the code or config? Did you understand what you have modified? No
  2. What dataset did you use? It's a nifti file, I just load the nifti file as ndarray and put array as input

Environment Windows 10

  1. Please run python mmdet/utils/collect_env.py to collect necessary environment information and paste it here.
  2. You may add addition that may be helpful for locating the problem, such as
    • How you installed PyTorch [e.g., pip, conda, source] conda
    • Other environment variables that may be related (such as $PATH, $LD_LIBRARY_PATH, $PYTHONPATH, etc.)

Error traceback

below Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "~\anaconda3\envs\Pytorch\Lib\site-packages\mmflow\apis\inference.py", line 131, in inference_model
    data = test_pipeline(data)
           ^^^^^^^^^^^^^^^^^^^
  File "~\anaconda3\envs\Pytorch\Lib\site-packages\mmflow\datasets\pipelines\compose.py", line 42, in __call__
    data = t(data)
           ^^^^^^^
  File "~\anaconda3\envs\Pytorch\Lib\site-packages\mmflow\datasets\pipelines\transforms.py", line 615, in __call__
    results[k] = mmcv.imnormalize(results[k], self.mean, self.std,
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~\anaconda3\envs\Pytorch\Lib\site-packages\mmcv\image\photometric.py", line 27, in imnormalize
    return imnormalize_(img, mean, std, to_rgb)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "~\anaconda3\envs\Pytorch\Lib\site-packages\mmcv\image\photometric.py", line 48, in imnormalize_
    cv2.subtract(img, mean, img)  # inplace
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cv2.error: OpenCV(4.9.0) D:\a\opencv-python\opencv-python\opencv\modules\core\src\arithm.cpp:650: error: (-209:Sizes of input arguments do not match) The operation is neither 'array op array' (where arrays have the same size and the same number of channels), nor 'array op scalar', nor 'scalar op array' in function 'cv::arithm_op'.

Bug fix If you have already identified the reason, you can provide the information here. If you are willing to create a PR to fix it, please also leave a comment here and that would be much appreciated!