search

fcjian / InstaGen

InstaGen: Enhancing Object Detection by Training on Synthetic Dataset, CVPR2024
https://fcjian.github.io/InstaGen
MIT License
68 stars 2 forks source link

Regarding training setup and GPU hours #6

Closed sunny1401 closed 1 month ago

sunny1401 commented 2 months ago

Hi - Thank you for this work. I had a question - I was going through the fine-tuning code. I see that you are using 8 GPUs. Could you tell how long it takes to train the model? Or maybe how long it takes to fine-tune the model on 8GPUs? I missed that information in the paper. I don't have access to 8 GPUs readily so it would help me looking for online solutions. for fine-tuning

fcjian commented 2 months ago

@sunny1401 We fine-tune Stable Diffusion on 8 A100, which takes approximately 6 hours.

sunny1401 commented 2 months ago

Thank you for that information

sunny1401 commented 2 months ago

Hi - I also wanted to ask - can you please explain your training setup. Because it's failing for me. What I mean by that is like the memory per GPU and if this was distributed training using multi-node setup.

fcjian commented 1 month ago

@sunny1401 We use single machine with 8 A100-80G.

sunny1401 commented 1 month ago

Oh thank you