How can I get the mapping of coordinate between stabilized_frame and frame?

[Ling-wei]

Thanks for sharing. I'm trying to find the location of stabilized_frame on the orignal input frame. Such as (x,y) -> (x',y'). Is there any possible to get the transformation? Thanks.

[AdamSpannbauer]

Hi @Ling-wei,

If you'd like to visualize the transformations used frame-to-frame you can use the VidStab.plot_transforms() method (usage is shown here in README). This method plots the VidStab.transforms attribute. The VidStab.transforms attribute is a 3 column numpy array that represents the frame-to-frame transformations used to stabilize the input video. The 3 columns, in order, represent the delta x, delta y, & delta angle (in radians).

[Ling-wei]

Hi @Ling-wei,

If you'd like to visualize the transformations used frame-to-frame you can use the VidStab.plot_transforms() method (usage is shown here in README). This method plots the VidStab.transforms attribute. The VidStab.transforms attribute is a 3 column numpy array that represents the frame-to-frame transformations used to stabilize the input video. The 3 columns, in order, represent the delta x, delta y, & delta angle (in radians).

Hi @AdamSpannbauer Thanks for replying! Yet, I have a few questions. I'm trying to plot a bounding box of orignal frame on the stabilized_frame but got the wrong position. So I wondered what is the rotation center of delta angle and the order of coordinate correction (first amend dx&dy or first rotate? ). If the border_size is not zero, should the mapping be modified?

[AdamSpannbauer]

I'll speak to the questions around transforms later in this reply, but if you're after the effect shown in the below gif then you can calculate the location of the box using the input frame dimensions and the chosen border size for the stabilization. The output seen in the gif was created with the code chunk below the gif.

import os
import cv2
from vidstab import VidStab, download_ostrich_video

BORDER_SIZE = 100
VIDEO_PATH = "ostrich.mp4"

# Download test video to stabilize
if not os.path.isfile(VIDEO_PATH):
    download_ostrich_video(VIDEO_PATH)

# Initialize stabilizer and video reader
stabilizer = VidStab()
vidcap = cv2.VideoCapture(VIDEO_PATH)

bb_p1 = bb_p2 = None
while True:
    grabbed_frame, frame = vidcap.read()

    # Calculate location of original frame in final output based on border size
    if frame is not None:
        h, w = frame.shape[:2]

        bb_p1 = (BORDER_SIZE, BORDER_SIZE)
        bb_p2 = (BORDER_SIZE + w, BORDER_SIZE + h)

    # Pass frame to stabilizer even if frame is None
    stabilized_frame = stabilizer.stabilize_frame(input_frame=frame, border_size=BORDER_SIZE)

    # If stabilized_frame is None then there are no frames left to process
    if stabilized_frame is None:
        break

    # stabilized_frame is returned as all 0 while warming up
    if stabilized_frame.sum() > 0:
        # Draw showing original frame location
        cv2.rectangle(stabilized_frame, bb_p1, bb_p2, (0, 255, 0), 2)

    cv2.imshow('Frame', stabilized_frame)
    key = cv2.waitKey(5)

    if key == 27 or key == ord('q'):
        break

vidcap.release()
cv2.destroyAllWindows()

---

The below code chunk shows an example of how frame-to-frame transformations are applied. This is done using cv2.warpAffine() (you can see some examples in OpenCV's documentation).

The code for the helper function build_transformation_matrix() can be seen here.

import cv2
import numpy as np
from vidstab.vidstab_utils import build_transformation_matrix

H = 300
W = 600
BORDER_SIZE = 100

transform = [
     50,           # delta x
    -30,           # delta y
    np.deg2rad(20) # delta angle
]

image = (np.random.rand(H, W) * 255).astype('uint8')
bordered_image = cv2.copyMakeBorder(image,
                                    top=BORDER_SIZE,
                                    bottom=BORDER_SIZE,
                                    left=BORDER_SIZE,
                                    right=BORDER_SIZE,
                                    borderType=cv2.BORDER_CONSTANT,
                                    value=0)

h, w = bordered_image.shape[:2]

transform_mat = build_transformation_matrix(transform)

transformed = cv2.warpAffine(bordered_image,
                             transform_mat,
                             (w, h),
                             borderMode=cv2.BORDER_CONSTANT,
                             borderValue=0)

cv2.imshow('Transformed', transformed)
cv2.waitKey(0)

[Ling-wei]

Hi @AdamSpannbauer , The problem has been solved. Thank you very much~