[Feature]: Support animated webp format

[dchatel]

Expected behavior

from webp import WebPAnimDecoder, WebPAnimDecoderOptions, WebPData

def _webp_reader(file):
    with open(file, 'rb') as f:
        webp_data = WebPData.from_buffer(f.read())
        dec_opts = WebPAnimDecoderOptions.new(use_threads=True)
        dec = WebPAnimDecoder.new(webp_data, dec_opts)

        frames_data = list(dec.frames())

    frames = [arr[:,:,:3] for arr, _ in frames_data]
    fps = 1000 * len(frames_data) / frames_data[-1][1]
    return frames, fps

[continue-revolution]

@zappityzap

[zappityzap]

@dchatel This looks like code to read a WebP video, but what is the request?

This extension uses the Gradio Video element to load source videos, and the Video element only appears to support loading MP4 and WebM.

[dchatel]

Sorry, the request would be to be able to load animated webp files as a video input.

I've used the gradio video element before to load animated webp. The animated webp is shown as a plain black image on the frontend, but you still can load it just fine on the backend. It's not "perfect" on the frontend, of course, but it gets the job done and it's better than nothing.

[zappityzap]

I see what you mean. scripts/animatediff_cn.py uses OpenCV in function get_input_frames() to get the frames from the video file, and it doesn't load animated WebP correctly. It should be possible to replace OpenCV with ImageIO for getting frames from videos, but it's not a high-priority for me.

2023-11-30 14:32:46,025 - AnimateDiff - WARNING - entered get_input_frames()
2023-11-30 14:32:46,025 - AnimateDiff - WARNING - get_input_frames: params.video_source = /tmp/gradio/7491ad3ecc79bc9d790de4b0ccbd1886fba93289/00000-1046048558.webp
[webp @ 0x7f0ccbe96ec0] invalid TIFF header in Exif data
[webp @ 0x7f0ccbe96ec0] image data not found
2023-11-30 14:32:46,027 - AnimateDiff - WARNING - cap opened
2023-11-30 14:32:46,027 - AnimateDiff - WARNING - reading a frame
[webp @ 0x7f0cc3ebeb80] invalid TIFF header in Exif data
[webp @ 0x7f0cc3ebeb80] image data not found
2023-11-30 14:32:46,049 - AnimateDiff - WARNING - cap released

[continue-revolution]

I have already implemented ffmpeg video extraction logic in another project. not sure if it will work for all the formats.

[continue-revolution]

there is ffmpeg right now but I'm not sure if it will work for webp animation. Will close for now. let me know if it's not working