lucidrains / deep-daze

Simple command line tool for text to image generation using OpenAI's CLIP and Siren (Implicit neural representation network). Technique was originally created by https://twitter.com/advadnoun
MIT License
4.37k stars 326 forks source link

Multiple GPU training #131

Open Uasmi opened 3 years ago

Uasmi commented 3 years ago

Hey,

I'm currently trying to make daze work on a multi gpu setup, however I don't have too much of experience working with pytorch. I tried to make a model call run in parallel by editing deep_daze.py at line 380 (also removing device mapping from model definition)

model = nn.DataParallel(model)
model.to(self.device)
self.model = model
self.scaler = GradScaler()
siren_params = model.module.parameters()

But I guess I am missing something with data mapping to devices, as I get error: TypeError: forward() missing 1 required positional argument: 'text_embed'