embed the text as well

[iejMac]

can't see why we shouldn't have an option for encoding the text and putting that in the WDS

[iejMac]

tasks:

• [ ] add optional encode_text parameter
• [ ] edit encode_chunk function (https://github.com/iejMac/clip-video-encode/blob/b14fbb57b464484d0443cfbc5c56a91bb52bbed3/clip_video_encode/clip_video_encode.py#L30) to encode the text if the encode_text parameter is True
• [ ] add the optional text_emb param to all writer.write methods (https://github.com/iejMac/clip-video-encode/blob/main/clip_video_encode/writer.py) and write same way as frames

[matchyc]

is that available now? I really need to encode text @iejMac

[iejMac]

you can't save the caption embeddings in the current state but you could probably hack around the code (or make a clean PR! that would be great) and get it to do that since they are computed here - https://github.com/iejMac/clip-video-encode/blob/ff390b01091e9d93263d3cb4e8a747fb4143cc1d/clip_video_encode/clip_video_encode.py#L107

currently they aren't saved but instead they are just compared to each frame in the input video to get a similarity score per frame and that is saved

[matchyc]

Thanks, I guess I'll be able to finish a pr in the future after getting my things done. By the way, is there embedded caption data for webvid? I only see video embeddings in huggingface.