Unable to set framerate for tonic.utils.plot_animation

[a-lohia]

Would it be possible to create a new function that just produces an mp4 video with the events with support for manual framerates. I currently have to do this manually using skvideo.io.FFmpegWriter()

def plot_animation(frames: np.ndarray, figsize=(5, 5), fps=120, save_path=None):
    """Helper function that animates a tensor of frames of shape (TCHW). If you run this in a
    Jupyter notebook, you can display the animation inline like shown in the example below.

    Parameters:
        frames: numpy array or tensor of shape (TCHW)
        figsize: tuple(int, int) specifying the size of the figure

    Returns:
        The animation object. Store this in a variable to keep it from being garbage collected until displayed.
    """
    try:
        import matplotlib.pyplot as plt
        from matplotlib import animation
    except ImportError:
        raise ImportError(
            "Please install the matplotlib package to plot events. This is an optional"
            " dependency."
        )
    fig = plt.figure(figsize=figsize)
    if frames.shape[1] == 2:
        rgb = np.zeros((frames.shape[0], 3, *frames.shape[2:]))
        rgb[:, 1:, ...] = frames
        frames = rgb
    if frames.shape[1] in [1, 2, 3]:
        frames = np.moveaxis(frames, 1, 3)

    frames = (frames / frames.max()) * 255

    print(frames.shape)

    writer = skvideo.io.FFmpegWriter(save_path, inputdict={"-r": str(fps)},
                                     outputdict={'-vcodec': 'libx264', '-r': str(fps)},
                                     verbosity=0)

    for i in tqdm(range(frames.shape[0]), desc="writing video"):
        writer.writeFrame(frames[i])

    # Release capture and destroy windows
    writer.close()

    return True

Thanks, A. L.

[biphasic]

Hello @a-lohia ! Wrapping a function like skvideo.io.FFmpegWriter has several pitfalls. First, the next person would like to select a different codec or change other parameters. That means we'd have to expose all the FFmpegWriter args in the wrapper function. Second, it's another dependency to Tonic, but writing videos is not the scope of Tonic.

What I normally do is to create frames using tonic.transforms.ToFrame, convert them to PyTorch tensors and then use torchvision's write_video https://pytorch.org/vision/stable/generated/torchvision.io.write_video.html

Would that be a solution for your problem?

[a-lohia]

Hello @a-lohia ! Wrapping a function like skvideo.io.FFmpegWriter has several pitfalls. First, the next person would like to select a different codec or change other parameters. That means we'd have to expose all the FFmpegWriter args in the wrapper function. Second, it's another dependency to Tonic, but writing videos is not the scope of Tonic.

What I normally do is to create frames using tonic.transforms.ToFrame, convert them to PyTorch tensors and then use torchvision's write_video https://pytorch.org/vision/stable/generated/torchvision.io.write_video.html

Would that be a solution for your problem?

That makes sense. I think your solution would work just as well.

Thank you for the quick reply, A.L.