Unclear why stabilizer.transforms is returning so many entries

[AlbertoMQ]

from vidstab import VidStab, layer_overlay, layer_blend
import cv2
import os

# init vid stabilizer
stabilizer = VidStab()

INPUT_FILE_PATH = ".../image_%06d.jpg"
OUT_FILE_PATH = ".../image_%06d.jpg"

stabilizer.stabilize(input_path=INPUT_FILE_PATH,
                     output_path=OUT_FILE_PATH,
                     border_type='black',
                     border_size= 0)

transforms = stabilizer.transforms
print("......", len(transforms))

Unclear why stabilizer.transforms is returning so many entries. This doesn't happen when border_size = 'auto'

Using input path "image_%06d.jpg" I am able to read and apply transforms. Why for an input of 48 images do I get len(transforms) = 78?

[AlbertoMQ]

I think it's coming from the 30 introduced in smoothing_window, unsure about the intuition behind this

[AdamSpannbauer]

Hi thanks for reporting. I’ll look into this tomorrow.

Would you please provide an example code snippet that you are using to make investigating and understanding your issue easier?

[AlbertoMQ]

Updated, notice the comment "This doesn't happen when border_size = 'auto'" Haven't messed around with these settings too much so maybe it works with other settings as well. I added a zip file with some images in another issue I created (#106 ).

[AdamSpannbauer]

Notes on issue

(I haven't had time to fully dive in yet, notes for when I do have time.)

The 'auto' option for border_size requires 2 passes of the input frames. It goes through once and creates all of the transforms (to try and guess reasonable border), and then it goes through again and applies the transforms.

The other border_size options of the stabilize method are processing in 1 pass. Go through just enough frames to meet the smoothing_window requirement and start applying transforms immediately (one pass, but the applying of transforms will be smoothing_window frames behind).

The issue must be somewhere in the stabilize_frame method's process. It needs to have more awareness of the end of the video stream to stop creating transforms. (perhaps if no change in frame_queue len?)

Contained reproducible example:

import tempfile
from vidstab import VidStab
import vidstab.download_videos as dl

TMP_DIR = tempfile.TemporaryDirectory()
VID_PATH = f'{TMP_DIR.name}/test_video.mp4'

SMOOTHING_WINDOW = 30
# Set to float("inf") to include all video
# issue exists with or without restricting frames
MAX_FRAMES = 32

dl.download_ostrich_video(VID_PATH)

stabilizer = VidStab()
stabilizer.stabilize(VID_PATH,
                     "stable.avi",
                     smoothing_window=SMOOTHING_WINDOW,
                     max_frames=MAX_FRAMES)

print("trajectory:")
print(stabilizer.trajectory)

print(f"n transforms: {stabilizer.transforms.shape[0]}")
print("nframes + SMOOTHING_WINDOW: "
      f"{min(stabilizer.frame_queue.source_frame_count, MAX_FRAMES) + SMOOTHING_WINDOW}")

[AdamSpannbauer]

Progress is limited to converting test to use stabilize method so that this issue causes tests to fail