Can this run on google colab? What is the minimum required VRAM? Any google colab notebook available?

[FurkanGozukara]

Can this run on free google colab?

What is the minimum required VRAM?

Any google colab notebook available?

I have recently a made a tutorial for stable diffusion 1.5 and dream booth. It works great. I would like to prepare a tutorial for this one too.

https://www.youtube.com/watch?v=mnCY8uM7E50

In this video, I will show you how to create your own Lensa app-like magic avatars without using any third-party apps. Our method ensures 100% privacy, unlike uploading your personal photos to external apps and platforms. I am sure you have seen many people sharing their magic avatars on social media. While these paid apps use free open-source AI models, they still require payment. Additionally, AI image generation apps may keep your private photos and use them as they please. Instead of paying and trusting the goodwill of these paid apps, we will use these open-source public AI models ourselves. I will provide step-by-step instructions, so even those with no technical expertise can follow along. We will be using Google's Colab, a free #AI platform that offers access to a powerful GPU at no cost. Therefore, this tutorial can even be completed on a mobile phone without a PC. All you need is a Gmail account. Once we train the open-source image generation model #StableDiffusion with #DreamBooth by using our own portrait images, the possibilities are endless. Unlike other applications, the method I will demonstrate does not impose any limitations on image generation. You will be able to freely compose any kind of image as many times as you desire. You can not only generate your own avatars, but also any other images you want, such as highly detailed car images or landscapes.

[nupurkmr9]

Hi,

Thanks a lot for your interest in our work. Currently we don't have a google colab. But we have a huggingface demo based on diffusers here: https://huggingface.co/spaces/nupurkmr9/custom-diffusion The minimum required VRAM for default setup is 30GB. But with 8-bit optimization and gradient checkpointing the training code based on diffusers library can be run on a T4 with 16GB VRAM.

[FurkanGozukara]

Hi,

Thanks a lot for your interest in our work. Currently we don't have a google colab. But we have a huggingface demo based on diffusers here: https://huggingface.co/spaces/nupurkmr9/custom-diffusion The minimum required VRAM for default setup is 30GB. But with 8-bit optimization and gradient checkpointing the training code based on diffusers library can be run on a T4 with 16GB VRAM.

i have rtx 3060 which has 12 gb vram. looks like i am out of option :/

if you make a google colab version or kaggle version that can run on free google colab or kaggle i would appreciate very much

by the way i am training my photos on t4 of google colab now. lets see what results i am gonna get

[2kpr]

Hi, Thanks a lot for your interest in our work. Currently we don't have a google colab. But we have a huggingface demo based on diffusers here: https://huggingface.co/spaces/nupurkmr9/custom-diffusion The minimum required VRAM for default setup is 30GB. But with 8-bit optimization and gradient checkpointing the training code based on diffusers library can be run on a T4 with 16GB VRAM.

i have rtx 3060 which has 12 gb vram. looks like i am out of option :/

if you make a google colab version or kaggle version that can run on free google colab or kaggle i would appreciate very much

by the way i am training my photos on t4 of google colab now. lets see what results i am gonna get

I ran custom diffusion fine on Windows with my 3080 Ti 12GB, it used anywhere from 9GB up to 12GB depending on my batch sizes, 2 for 9GB, 6 for 12GB, settings.

Also, If you train with --freeze_model=crossattn it outputs a 172MB delta.bin file vs the 75MB as it uses more vectors in the attention layer.

Settings I used:

subprocess.run(["accelerate",
                "launch",
                "custom-diffusion/src/diffuser_training.py",
                "--pretrained_model_name_or_path=" + MODEL_NAME,
                "--instance_data_dir=" + INSTANCE_DIR,
                "--class_data_dir=" + CLASS_DIR,
                "--output_dir=" + OUTPUT_DIR,
                "--instance_prompt='photo of a <new1> woman'",
                "--class_prompt='photo of a woman'",
                "--revision=fp16",
                "--mixed_precision=fp16",
                "--with_prior_preservation",
                "--prior_loss_weight=1.0",
                "--resolution=512",
                "--train_batch_size=4",
                "--train_text_encoder",
                "--gradient_checkpointing",
                "--use_8bit_adam",
                "--gradient_accumulation_steps=1",
                "--learning_rate=5e-6",
                "--lr_scheduler=constant",
                "--lr_warmup_steps=0",
                "--num_class_images=" + str(CLASS_IMAGES),
                "--sample_batch_size=1",
                "--max_train_steps=" + str(STEPS),
                "--scale_lr",
                "--modifier_token='<new1>'",
                "--freeze_model=crossattn"
                ])

[hamzafar]

Hi,

I am finetuning with multiple concepts with the following code on my Google Collab account:

The code initially worked fine as it has created images and saved them in the respective classes (concepts) directory. However it has given following error: OSError: Can't load the configuration of ''. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure '' is the correct path to a directory containing a config.json file

I know I may be missing some configuration part however any recommendation on this will be highly appreciated.

[hamzafar]

Hi,

I am finetuning with multiple concepts with the following code on my Google Collab account:

The code initially worked fine as it has created images and saved them in the respective classes (concepts) directory. However it has given following error: OSError: Can't load the configuration of ''. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure '' is the correct path to a directory containing a config.json file

I know I may be missing some configuration part however any recommendation on this will be highly appreciated.

After spending couple of hours, I have resolved this issue. The only bug was version mismatch of transformer.

[arushijain71]

Hi,

I am finetuning with multiple concepts with the following code on my Google Collab account:

The code initially worked fine as it has created images and saved them in the respective classes (concepts) directory. However it has given following error: OSError: Can't load the configuration of ''. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure '' is the correct path to a directory containing a config.json file

I know I may be missing some configuration part however any recommendation on this will be highly appreciated.

Hi @hamzafar can you please share your colab notebook, as I am trying to train it for multi concepts and I am getting memory error.

[FurkanGozukara]

Hi, I am finetuning with multiple concepts with the following code on my Google Collab account: The code initially worked fine as it has created images and saved them in the respective classes (concepts) directory. However it has given following error: OSError: Can't load the configuration of ''. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure '' is the correct path to a directory containing a config.json file I know I may be missing some configuration part however any recommendation on this will be highly appreciated.

Hi @hamzafar can you please share your colab notebook, as I am trying to train it for multi concepts and I am getting memory error.

How to Do SDXL Training For FREE with Kohya LoRA - Kaggle - NO GPU Required - Pwns Google Colab