TypeError: 'NoneType' object is not iterable

[korabelnikov]

Describe the bug A crash

Provide version info python 3.6 opencv 4.1 master

Provide error message

 File "/home/user/anaconda3/envs/mmdet2/lib/python3.7/site-packages/vidstab/VidStab.py", line 516, in stabilize
   output_fourcc=output_fourcc, progress_bar=bar)
 File "/home/user/anaconda3/envs/mmdet2/lib/python3.7/site-packages/vidstab/VidStab.py", line 214, in _apply_transforms
   self._gen_next_raw_transform()
 File "/home/user/anaconda3/envs/mmdet2/lib/python3.7/site-packages/vidstab/VidStab.py", line 116, in _gen_next_raw_transform
   matched_keypoints = vidstab_utils.match_keypoints(optical_flow, self.prev_kps)
 File "/home/user/anaconda3/envs/mmdet2/lib/python3.7/site-packages/vidstab/vidstab_utils.py", line 70, in match_keypoints
   for i, matched in enumerate(status):
TypeError: 'NoneType' object is not iterable

Provide code snippet

if not os.path.exists(folder_out):
    os.makedirs(folder_out)

for name in os.listdir(folder):
    path_in = os.path.join(folder, name)
    path_out = os.path.join(folder_out, name)
    assert os.path.exists(path_in)
    stabilizer = VidStab()
    stabilizer.stabilize(input_path=path_in, output_path=path_out,
                         #playback=True,
                         show_progress=True,
                         output_fourcc='mp4v'
                         )

Are you able to provide the video? I able if you need, just mention about it

---

Edited by @AdamSpannbauer for some formatting stuff.

[AdamSpannbauer]

Hi @korabelnikov, thanks for reporting this issue and #95. I have a pretty full schedule this week but I should be able to investigate this upcoming weekend.

[AdamSpannbauer]

Hi @korabelnikov, unfortunately, I've unable to reproduce this issue so far on my own. Would you be willing to share your video that's running into this issue? If you're willing, you can send me the video/a link to the video at my personal email address: spannbaueradam@gmail.com.

From your error message, I can tell that the problem is cv2.calcOpticalFlowPyrLK() is returning a None status (the function returns a tuple of (points, status, error)).

The problem is, I don't know the case where status is None, everything I've tried results in cv2.calcOpticalFlowPyrLK() throwing an error rather than returning a None status. Hopefully, I can recreate the issue with your video and catch the exception (or throw a more informative one)

[korabelnikov]

I've sent a video via email

[AdamSpannbauer]

Thanks, I've received the video. I should be able to get into this over the weekend.

[korabelnikov]

a patch i used to fix it up

Index: vidstab_utils.py
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
--- vidstab_utils.py    (date 1577795342819)
+++ vidstab_utils.py    (date 1577795342819)
@@ -67,7 +67,8 @@
     # storage for keypoints with status 1
     prev_matched_kp = []
     cur_matched_kp = []
-    
+    if status is None:
+        return [], []
     for i, matched in enumerate(status):
         # store coords of keypoints that appear in both
         if matched:

[AdamSpannbauer]

Awesome! I still want to investigate what the cause is for the None status, but if we're unable to figure that out, then the next best thing would be to treat the symptom like you've done here.

Would you be willing to open up a PR? If we can't figure out the None status (and this doesn't cause any unit tests to fail) I would be very open to merging your work.

[AdamSpannbauer]

Below is the culprit code, frames, and keypoints that led to the error when processing the video you shared with me on my machine.

Line of code that led to the None status

optical_flow = cv2.calcOpticalFlowPyrLK(self.prev_gray,
                                        current_frame_gray,                                                
                                        self.prev_kps, None)

Contents of self.prev_kps:

np.array([], shape=(0, 1, 2), dtype=float32)

Contents of self.prev_gray:

Contents of current_frame_gray:

[AdamSpannbauer]

More encapsulated way to reproduce the bug:

import cv2
import numpy as np
from vidstab import vidstab_utils as utils

prev_gray = np.ones((1208, 1920, 3)) * 250
current_frame_gray = np.ones((1208, 1920, 3)) * 250
prev_kps = np.array([], dtype='float32')
prev_kps = prev_kps.reshape(0, 1, 2)

optical_flow = cv2.calcOpticalFlowPyrLK(prev_gray,
                                        current_frame_gray,
                                        prev_kps,
                                        None)

matched_keypoints = utils.match_keypoints(optical_flow, prev_kps)
transform_i = utils.estimate_partial_transform(matched_keypoints)

[AdamSpannbauer]

Merging #100 should fix the issue. @korabelnikov I essentially added your proposed changes. Thanks for reporting the issue and proposing the changes.

[korabelnikov]

great!