The processed subtitles by Promptfactory are not sent to the video model with PromptBasedVideo

[leclem]

The processed subtitles by Promptfactory are not sent to the video model. Instead, it just sends the text

Reproduce :

@title Generating a video from simple text prompt

working_folder="./examples/inputs/PromptbasedVideo/" with WorkingFolderContext(working_folder): video_build_settings = VideoBuildSettings( music_building_context=MusicBuildingContext( apply_background_music=True, generate_background_music=True, ), test_mode=False, include_read_aloud_prompt=True, target_model_provider="videocrafter", output_video_file_name="AICosmetics.mp4", interpolate=True, )

prompt = "Unlock your radiance with AI Cosmetics. Experience the magic of premium ingredients, designed to reveal your natural glow. "  # @param {type:"string"}
gw = video_build_settings.get_ml_models_gateway()
prompt = await PromptFactory(ml_gateway=gw).create_prompt_from_text(prompt)
video = PromptBasedVideo(prompt=prompt)
await video.build(build_settings=video_build_settings)

In the logs : 2024-08-04 19:43:11.603 | DEBUG | vikit.gateways.vikit_gateway:generate_video_VideoCrafter2_async:721 - Generating video from prompt: Block your radiance with AI cosmetics. Use the magic of premium ingredients, designed to reveal your natural glow.

[jeffmac-aix]

Reproduced on a copied notebook, first the subtitles are 9 seconds long, so we should have 2 subs after merge, but it seems we got a single one, indeed. Did another test with longer prompt and saw another strange issue: the srt generated has a crazy length for the latest subtitle item, although file name shows the right length 1 00:00:00,000 --> 00:00:02,960 Lock your radiance with AI cosmetics.

2 00:00:02,960 --> 00:00:08,240 Experience the magic of premium ingredients, designed to reveal your natural glow.

3 00:00:08,240 --> 00:00:11,880 Embrace the future of beauty, tailored, just for you.

4 00:00:11,880 --> 00:00:41,840 Discover a new level of confidence with personalized skincare.

[jeffmac-aix]

I got 4 composite generated, one per subtitle, which is expected, so there seems not to be any issue with the list of subtitles, unit test test_trainboy_moss_stones_prompt shows we do have the expected number of subs too. Let's dive deeper on your issue to see if there is still one, and understand why we have a crazy duration for latest sub.