search

upes-open / OSoc-24-The-Content-Forge

The Content Hub Is a online platform which acts as a all in one solution helping content creators develop and generate short form video image content utilising genai models and cloud to maximize their efficiency and benefit from the ever-growing developments in ai models
GNU Affero General Public License v3.0
2 stars 8 forks source link

Integrate AI Model for Video Generation via HuggingFace API #5

Open Manav-Khandurie opened 3 months ago

Manav-Khandurie commented 3 months ago

Use the HuggingFace API to run open source Video Models and integrate them to the Image/Video Generation Microservice

Ensure that there is proper video generation with the model that was chosen , implement the model with lowest running cost possible

Ensure that the video generated can be of 15-30s long and provide the functionality of goin Text --> Video.

The generated videos must be stored in the S3 Bucket , whose credentials will be provided .

nandiniiiiiiii commented 3 months ago

sir in this we have to create a model that takes text and convert it to video right....?? using hugging face api and implement it via react??

Manav-Khandurie commented 3 months ago

sir in this we have to create a model that takes text and convert it to video right....?? using hugging face api and implement it via react??

Yes @nandiniiiiiiii you are almost correct so in this issue basically we need to integrate some text to video from hugging face via its api , but this is done on the backend microservice ,so we have a node/python microservice that accepts an api request with text init and the service communicate with hugging face api to run any free model on hugging face platform and then genrate and send it back .

We can store the image/video generated in a s3 buckets as well

nandiniiiiiiii commented 3 months ago

Oooo ok, actually I have worked with hugging face and implemented it using Flask so I thought it was something similar, but it looks a little more complex thankyou so much for clarifying it.

Manav-Khandurie commented 3 months ago

Oooo ok, actually I have worked with hugging face and implemented it using Flask so I thought it was something similar, but it looks a little more complex thankyou so much for clarifying it.

No worries @nandiniiiiiiii , if you change your mind, and wanna do this issue just let me know

adx04 commented 2 months ago

hello there @Manav-Khandurie ,i want on work on this issue this one looks pretty interesting..can you assign it to me?

adx04 commented 2 months ago

hello there @Manav-Khandurie ,i want on work on this issue this one looks pretty interesting..can you assign it to me?

hello !@Manav-Khandurie ,in which folder i have to work..pls guide me a little and can you mention the S3 bucket and API credentials?

Manav-Khandurie commented 2 months ago

hello there @Manav-Khandurie ,i want on work on this issue this one looks pretty interesting..can you assign it to me?

Are you still interested @adx04 sorry for the late response

adx04 commented 2 months ago

hello there @Manav-Khandurie ,i want on work on this issue this one looks pretty interesting..can you assign it to me?

Are you still interested @adx04 sorry for the late response

yes @Manav-Khandurie ! pls guide me a liittle on this issue like about frameworks and credentials for the s3 ?

rampantvoid commented 2 months ago

@Manav-Khandurie hi can I be assigned on this?

Manav-Khandurie commented 2 months ago

@Manav-Khandurie hi can I be assigned on this?

ok

Manav-Khandurie commented 2 months ago

hello there @Manav-Khandurie ,i want on work on this issue this one looks pretty interesting..can you assign it to me?

Are you still interested @adx04 sorry for the late response

yes @Manav-Khandurie ! pls guide me a liittle on this issue like about frameworks and credentials for the s3 ?

Sorry for the late response , @adx04 , actually you would be integrating the models form your ac count and just setting up the .env variables with hugging face , s3 upload functionality is done so leave that aspect now

rampantvoid commented 2 months ago

@Manav-Khandurie hey I was working on this and it seems we cannot run a text-to-video or any diffuser model off the hugging face api, they're not yet supported. Already have pinged you regarding this on discord