Is it possible to train the model using more than 4 GPUs and also with distributed computing ?

AlexeyAB / darknet

YOLOv4 / Scaled-YOLOv4 / YOLO - Neural Networks for Object Detection (Windows and Linux version of Darknet )

http://pjreddie.com/darknet/

Other

21.56k stars 7.95k forks source link

Open surajitkundu29 opened 4 years ago

surajitkundu29 commented 4 years ago

For Ex. I have two GPU enabled server and both having 4 GPUs, How my training can be distributed over these two server ?

AlexeyAB commented 4 years ago

No. There isn't implemented the training of one model on several servers.

without synchronized-batch-normalization it will hurt accuracy - training on more than 4 x GPUs
with synchronized-batch-normalization it will hurt training speed - training on several servers (but you will be able to use large mini_batch_size that will increase accuracy) - instead for this I suggest to use GPU-processing on CPU-RAM: https://github.com/AlexeyAB/darknet/issues/4386