Great work. I currently have millions of data that need to be trained. I would like to ask if it supports multi-gpu training?

NormXU / nougat-latex-ocr

Codebase for fine-tuning / evaluating nougat-based image2latex generation models

https://arxiv.org/abs/2308.13418

Apache License 2.0

115 stars 13 forks source link

Great work. I currently have millions of data that need to be trained. I would like to ask if it supports multi-gpu training? #7

Closed pjpanadas closed 1 month ago

pjpanadas commented 2 months ago

Or what needs to be changed to support multi-gpu training? I look forward to your reply.

NormXU commented 2 months ago

@pjpanadas Yes, the codebase supports DDP on a single node using the accelerate. If you have multiple nodes, I think you need to make some modifications.

pjpanadas commented 2 months ago

@pjpanadas Yes, the codebase supports DDP on a single node using the accelerator. If you have multiple nodes, I think you need to make some modifications.

Could you please tell me what needs to be changed? I am new to this field and I don't know what needs to be changed. Thank you very much.

NormXU commented 2 months ago

@pjpanadas This tutorial can be helpful. I believe you need to at least update the accelerate configuration. Could you let me know how many nodes you have?