cloneofsimo / lora

Using Low-rank adaptation to quickly fine-tune diffusion models.
https://arxiv.org/abs/2106.09685
Apache License 2.0
7.01k stars 480 forks source link

Data to reproduce the results #65

Open VladAndronik opened 1 year ago

VladAndronik commented 1 year ago

I can't get good results fine-tuning on faces, maybe there is a bug there, would be good to have data to reproduce your results from Readme with your training settings

cloneofsimo commented 1 year ago

I think this is crucial. Hold on I am making them.

Thomas-MMJ commented 1 year ago

So Joe Penna (MysteryGuitarMan on Discord), (one of the early adopters of dreambooth) has stated that he is willing to have his image set as a validation/comparison set for training on people.

Here is the images he has uploaded for usage,

https://drive.google.com/drive/folders/1k3BMK3tmrHwjQYGzTVQ8cH4eZGzK5UJ_

Thomas-MMJ commented 1 year ago

Here are the original corgi photos

https://unsplash.com/photos/9M0tSjb-cpA https://unsplash.com/photos/bQaAJCbNq3g https://unsplash.com/photos/brFsZ7qszSY https://unsplash.com/photos/eoqnr8ikwFE https://unsplash.com/photos/Id1DBHv4fbg

Yellow clock https://unsplash.com/photos/pxpw52_Cvsg https://unsplash.com/photos/zp5EGN2vxLM https://unsplash.com/photos/-XNtAXpXubM https://unsplash.com/photos/t6hNUc8vspA https://unsplash.com/photos/vcb0zyfS758 https://unsplash.com/photos/1VIyN3kq14I

red backpack https://unsplash.com/photos/5FffGvuAq6U https://unsplash.com/photos/pRtACvPy8XU https://unsplash.com/photos/qCWltV_QJ4M https://unsplash.com/photos/S1aYxHvpqxY https://unsplash.com/photos/9-9ezEUPrPc https://unsplash.com/photos/KGq78h_aO2E https://unsplash.com/photos/hHwvAcIS3eI https://unsplash.com/photos/xihj0AanfBI https://unsplash.com/photos/McufHC16mNg https://unsplash.com/photos/9w8FA2Wwe9s https://unsplash.com/photos/4s7TZBq0N5Y

vase https://unsplash.com/photos/lh7ZFoTtfi0 https://unsplash.com/photos/0dg-EfHgA8Q https://unsplash.com/photos/j5JvgCa1uIk

teapot https://unsplash.com/photos/z26H7EPwARg https://unsplash.com/photos/9kk22G44kvg https://unsplash.com/photos/VloyvGLX_kU https://unsplash.com/photos/_Ycnch2Ez5o https://unsplash.com/photos/SSj5qtnrpj0

cat toy https://huggingface.co/sd-dreambooth-library/cat-toy

license https://unsplash.com/license

and the demo using the above images in the original dreambooth paper

https://dreambooth.github.io/

Here are all of the textual inversion datasets https://drive.google.com/drive/folders/1d2UXkX0GWM-4qUwThjNhFIPP7S6WUbQJ

cloneofsimo commented 1 year ago

Oh wow this is great @Thomas-MMJ ! Thank you so much!

cloneofsimo commented 1 year ago

So these can be used freely right? I will add these into the repo

Thomas-MMJ commented 1 year ago

The textual inversion data sets I'm not certain what the license is on those. The unsplash license is extremely liberal, they summarize it thus,

Unsplash photos are made to be used freely. Our license reflects that.

All photos can be downloaded and used for free
Commercial and non-commercial purposes
No permission needed (though attribution is appreciated!)

What is not permitted 👎
Photos cannot be sold without significant modification.
Compiling photos from Unsplash to replicate a similar or competing service.

So should be fine to include them, but as suggested I'd credit both the site, and the photographers (photographer is at each link, images from the same batch are by the same photographer).

Joe Penna didn't give a specific license, but they are freely distributable and usable for the purposes of training and comparing.

The 'Textual Inversion' I'm not sure what license they are under, I think they are unsplash images but haven't looked them up, so not positive.

Thomas-MMJ commented 1 year ago

Note on the Textual Inversion site, they state,

Datasets which appear in the paper are being uploaded
 [here](https://drive.google.com/drive/folders/1d2UXkX0GWM-4qUwThjNhFIPP7S6WUbQJ). 
Some sets are unavailable due to image ownership. We will upload more as we recieve permissions to do so.

Pretained models coming soon.

So it is unclear what license they are under, The TI paper writers have stated they have permission to distribute them, but I don't know if others do, so I'd just link to their dataset, rather than include the images in your repository.

cloneofsimo commented 1 year ago

Nice!! ok thank you @Thomas-MMJ !!