Thumbnails vtt file generation for Bunny CDN

[piszczu4]

I started to use Bunny CDN to host my video files. They produce several sprite files for a given video like below:

Is it possible to somehow produce a vtt file that is compatible with vidstack player to use these thumbnails?

[Curetix]

Sure, you can link to multiple images in the VTT, something like this:

WEBVTT

00:00:00.000 --> 00:00:30.000
yourdomain.com/yourpath/_0.jpg#xywh=0,0,284,160

00:01:00.000 --> 00:01:30.000
yourdomain.com/yourpath/_0.jpg#xywh=284,0,284,160

...

00:10:00.000 --> 00:10:30.000
yourdomain.com/yourpath/_1.jpg#xywh=0,0,284,160

...

If you know time interval between the sprites and their dimensions, it shouldn't be difficult to generate.

Ref: https://www.vidstack.io/docs/player/core-concepts/loading#vtt

[piszczu4]

@Curetix yeah that's the problem, i.e. knowing exact sprite intervals and dimensions. I know that MUX generates VTT but Bunny does not and I'm trying to somehow figure out these required informations to produce VTT on my own

[Curetix]

Let's do some math to figure these out.

The individual image files contain a 6x6 grid of sprites, except probably the last one, but I can't fully tell from your screenshot. So to get the total amount of sprites: spriteCount = imageCount * 6 * 6 - (6 * 6 - spritesOnLastImage).

From that you can calculate the time interval between sprites using the duration of the video in seconds: interval = Math.ceil(videoDuration / spriteCount). I'm rounding here since Bunny likely uses a nice round number as their interval, while our videoDuration might be off by a few (milli)seconds.

As for the sprite dimensions, take the dimensions of an image (_0.jpg for example) and divide them by six: spriteWidth = imageWidth / 6; spriteHeight = imageHeight / 6.

Bunny probably uses the same time intervals and image/sprite dimensions for all videos, so you only to calculate these numbers once. Then you can generate a thumbnail VTT or JSON for a video using it's duration.

[piszczu4]

Sprite dimensions are different (one movie has 1800x1350 while another has 1800x1008). Last sprite looks strange as well so calculatng spritesOnLastImage might be hard:

In general I will ask their support why can't they produce these vtt files

[Bennycopter]

Hi there @piszczu4. I ran into this same issue with BunnyCDN. Here's some code to generate VTT files for Bunny's thumbnails.

All of our videos are 16:9 aspect ratio, but it sounds like you have some that are 4:3. This is what's affecting the thumbnail sheet size to be different for you (16:9 videos have 1800x1008 thumbnail sheets; 4:3 videos have 1800x1350 thumbnail sheets). You will need to keep track of that on your end and pass that data into this function.

There are three other properties that are required (thumbnailCount, length, guid), but you can query Bunny's API for those.

const BUNNY_CDN_HOSTNAME = "....."; // whatever your CDN Hostname is from Bunny

function generateSkimThumbnailVtt(video: {
    // data from Bunny's API - https://docs.bunny.net/reference/video_getvideo
    thumbnailCount: number,
    length: number,
    guid: string,
    // data NOT from Bunny
    aspectRatio: "4:3"|"16:9",
}): string {
    const ROWS = 6;
    const COLS = 6;
    const FRAMES_PER_PAGE = ROWS * COLS;
    const FRAME_WIDTH = 300;
    const FRAME_HEIGHT = video.aspectRatio === "4:3" ? 225 : 168.75;

    const segments = ["WEBVTT"];

    const frameDuration = video.length / video.thumbnailCount;

    for (let frame = 0; frame < video.thumbnailCount; frame++) {
        const startTime = frame * frameDuration;
        const endTime = startTime + frameDuration;
        const pageNum = Math.floor(frame / FRAMES_PER_PAGE);
        const pageUrl = `https://${BUNNY_CDN_HOSTNAME}/${video.guid}/seek/_${pageNum}.jpg`;
        const frameX = frame % COLS;
        const frameY = Math.floor((frame % FRAMES_PER_PAGE) / ROWS);

        segments.push(
            // e.g.
            //00:00:07.000 --> 00:00:08.000
            //https://path/to/image.jpg#xywh=1260,0,180,101
            `${formatVttTimestamp(startTime)} --> ${formatVttTimestamp(endTime)}` + "\n" +
            `${pageUrl}#xywh=${frameX*FRAME_WIDTH},${frameY*FRAME_HEIGHT},${FRAME_WIDTH},${FRAME_HEIGHT}`
        )
    }

    return segments.join("\n\n");
}

function formatVttTimestamp(timeInSeconds: number): string {
    // I used Day.js for this function, but you can write your own if you don't use Day.js
    // INPUT = Time in seconds
    // OUTPUT = e.g. "00:12:34.567"
    const HmsPart = dayjs.duration(timeInSeconds, "seconds").format("HH:mm:ss");
    const msPart = (timeInSeconds % 1).toFixed(3).slice(1); // turns 3.14159 into ".142"
    return HmsPart + msPart;
}

[bruecksen]

@Bennycopter thanks for your generateSkimThumbnailVtt thats exactly what I needed. Still at the beginning of switching to vidstack and wondering how do I set the thumbnails to the return of the method? Is there a way to set this programmatically after the initialization of the player?

[Bennycopter]

@bruecksen Sure thing! I'm happy it helps.

On my site, I have an endpoint for generating thumbnail .vtt files on the fly. The response from this endpoint is just the output from generateSkimThumbnailVtt. So this is what I do:

document.querySelector("media-video-layout").thumbnails = // the URL to the endpoint, e.g. `/generated/video-thumbnails.vtt?id=${videoId}`;

The Vidstack documentation describes how to set it programmatically in other ways, like by generating ThumbnailImageInit and ThumbnailStoryboard objects. I haven't tried those, though.

[bruecksen]

@Bennycopter thank you! That helps a little. But it seems only to work if an endpoint returns a vtt file. Programmatically I cant get it working with multiple sprite images.ThumbnailStoryboard seems to only work for a single image but not on multiple images? Or am I missing something here? And ThumbnailImageInit has no ability to work with sprites right?

[Bennycopter]

Hi @bruecksen, it seems that ThumbnailImageInit can work with sprites (via width, height, coords.x, and coords.y). If you use an IDE that has proper autocomplete, you can see what properties are available while writing code.

In case you don't have autocomplete, here's the type definition:

interface ThumbnailImageInit {
    url: string | URL;
    startTime: number;
    endTime?: number;
    width?: number;
    height?: number;
    coords?: ThumbnailCoords;
}
interface ThumbnailCoords {
    x: number;
    y: number;
}