Model Request: Addition of new Lava-1.6 models

jhc13 / taggui

Tag manager and captioner for image datasets

GNU General Public License v3.0

658 stars 30 forks source link

Model Request: Addition of new Lava-1.6 models #50

Closed Enyakk closed 5 months ago

Enyakk commented 7 months ago

As seen here: https://llava-vl.github.io/blog/2024-01-30-llava-1-6/ https://huggingface.co/liuhaotian/llava-v1.6-vicuna-13b/tree/main

BenDes21 commented 7 months ago

agreeee

jhc13 commented 7 months ago

I see there's a lot of demand for new models, including this one and #49, but I am currently very busy and do not have the time to work on adding them. I will have more free time in a few days, so I will take a look then.

BenDes21 commented 7 months ago

I see there's a lot of demand for new models, including this one and #49, but I am currently very busy and do not have the time to work on adding them. I will have more free time in a few days, so I will take a look then.

Thanks

Goldenkoron commented 7 months ago

I would like to see support for moondream added, unsure if it's already compatible or not if I dragged the model somewhere.

jhc13 commented 7 months ago

I would like to see support for moondream added, unsure if it's already compatible or not if I dragged the model somewhere.

Please create a separate issue for this. Thank you.

jhc13 commented 6 months ago

It seems like LLaVA-1.6 might be officially supported in Transformers soon (there's a pull request in progress for it), so I will wait for now.

BenDes21 commented 6 months ago

It seems like LLaVA-1.6 might be officially supported in Transformers soon (there's a pull request in progress for it), so I will wait for now.

nice cannot wait

Vigilence commented 6 months ago

Been doing some testing and this model can give impressive results. It can sometimes not detail items that are always detailed in cog va, but I think the 34b model might fix that.

BenDes21 commented 6 months ago

Been doing some testing and this model can give impressive results. It can sometimes not detail items that are always detailed in cog va, but I think the 34b model might fix that.

cannot wait to test, is it less vram consuming than cog va ?

Vigilence commented 6 months ago

The ram used depends on the model. For:

the 34gb model my gpu is using nearly all 24gb vram sometimes a little past it.
the 13gb model uses around 20gb of vram
the 7gb model uses around 11gb vram.

I am sure these models could use less vram, but the gui i'm using to access these models may not be fully optimized. All models are using 4-bit mode. This model is good for art identification, at least for me, as it points out many little details the other models won't point out. Not even the top rated ones like Qwen-vl-max.

LoFiApostasy commented 5 months ago

This would be great, v1.6 is pretty good. It looks like the pr has changed to here: https://github.com/huggingface/transformers/pull/29586 and is in the final stages of approval.

jhc13 commented 5 months ago

The models have been added in v1.19.0.

Vigilence commented 5 months ago

The models have been added in v1.19.0.

Thank you!