search

jhc13 / taggui

Tag manager and captioner for image datasets
GNU General Public License v3.0
659 stars 30 forks source link

Feature request: JoyCaption as captioner #263

Open StableLlama opened 4 weeks ago

StableLlama commented 4 weeks ago

As seen at https://huggingface.co/spaces/fancyfeast/joy-caption-pre-alpha it is a model with superb captioning skills. It output was nearly creating the test image in Flux again. So it would be great when it could be used as a taggui autocaptioner.

The implementation could be copied from the spaces as linked above.

StableLlama commented 4 weeks ago

It seems this is the ordinary meta-llama/Meta-Llama-3.1-8B with the LoRA https://huggingface.co/spaces/fancyfeast/joy-caption-pre-alpha/blob/main/wpkklhc6/image_adapter.pt on top

StableLlama commented 2 weeks ago

There's a fork of taggui that includes it now: https://github.com/doloreshaze337/taggui

ArmadstheDoom commented 2 weeks ago

There's a fork of taggui that includes it now: https://github.com/doloreshaze337/taggui

This doesn't actually seem to work? Following the instructions on that fork, it simply doesn't recognize the existence of the joycaption folder or the included imageadpter.pt. I have no idea why it doesn't, but it doesn't. Doesn't matter if you use the release or a git pull version, neither recognize the existence of it, so right now you can't use joycaption. Which sucks, because it's essentially required for tagging things for FLUX.

StableLlama commented 2 weeks ago

This doesn't actually seem to work?

When it's not working you should open up an issue there and not here.

ArmadstheDoom commented 2 weeks ago

This doesn't actually seem to work?

When it's not working you should open up an issue there and not here.

The reason I put it here is because this is a feature request, and I didn't want them to go 'well there's a fork so no need to add it to the main branch.' I think that this is something that should be added to the main branch support since it drastically improves tagging for FLUX models.

jhc13 commented 2 weeks ago

I'll look into adding it as soon as I have time (I'm not sure when that will be).

ArmadstheDoom commented 2 weeks ago

I'll look into adding it as soon as I have time (I'm not sure when that will be).

Glad to hear! Obviously, when you do it for free, one can't demand it be immediate. But I'm glad to hear it'll be added. Take your time though!