search

LLaVA-VL / LLaVA-NeXT

Apache License 2.0
2.3k stars 154 forks source link

[Question] Is this slated for release in the Transformers library? #9

Open WAS-PlaiLabs opened 3 months ago

WAS-PlaiLabs commented 3 months ago

Is this going to be on the transformers library? Seems like it's going to be big.

zucchini-nlp commented 3 months ago

Hey! Yes, we would love to have it in the library 🤗

I am planning to work on it. Otherwise, @Luodian @ZhangYuanhan-AI let me know if you want to contribute it yourselves, I will be happy t help 😄

Luodian commented 3 months ago

Hey! Yes, we would love to have it in the library 🤗

I am planning to work on it. Otherwise, @Luodian @ZhangYuanhan-AI let me know if you want to contribute it yourselves, I will be happy t help 😄

Hi we still have more release to be done and we are working hard for it. So can we wait for sometime, maybe next month, we can do this PR to update LLaVA-NeXT to huggingface transformers! Looking forward to doing it!

zucchini-nlp commented 3 months ago

Great, will be looking forward for the next release. Let me know if you need any guidance for contributing a model 🤗

zucchini-nlp commented 3 months ago

@Luodian Hey again! Just wanted to check in and see if you had any updates on this. Thanks!

Luodian commented 3 months ago

@Luodian Hey again! Just wanted to check in and see if you had any updates on this. Thanks!

Hi! Thanks for your interests, we plan to focus on two or three weeks later (with more changes), is that a good time for you? or will break your progress. If so, before that, you could implement current llava-next-video. Then we could further contribute on it.

zucchini-nlp commented 3 months ago

@Luodian I see, thanks. Implementing the current state of "llava-next-video" sounds good for me. The model shows very good performance on videos, and we can give it more visibility on the hub by adding it to transformers

Also, if the new release is similar to current in terms of modeling/processing code, updating further will be easier and faster

Luodian commented 3 months ago

yes, totally agree!

zucchini-nlp commented 2 months ago

The model is added to Transformers and will be part of the next 4.42 release! Please find all checkpoints here: https://huggingface.co/collections/llava-hf/llava-next-video-6666a9173a64c7052930f153 🤗