search

yjh0410 / yolov2-yolov3_PyTorch

Apache License 2.0
223 stars 53 forks source link

Why is training so slow #34

Open lilslsls opened 4 years ago

lilslsls commented 4 years ago

4900 images and batchsize：64 gpu：2080TI

yjh0410 commented 4 years ago

Because the workers in dataloader is 0. I denote this in my train code. If I add workers, it will report some errors when I use multiscale training trick. I have no idea how to solve this problems. 

---Original--- From: "lilslsls"<notifications@github.com> Date: Sat, May 23, 2020 13:54 PM To: "yjh0410/yolov2-yolov3_PyTorch"<yolov2-yolov3_PyTorch@noreply.github.com>; Cc: "Subscribed"<subscribed@noreply.github.com>; Subject: [yjh0410/yolov2-yolov3_PyTorch] Why is training so slow (#34)

4900 images and batchsize：64 gpu：2080TI

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

yjh0410 commented 4 years ago

And if you don't plan to try multi scale training, you'd better set num_workers as 8 or more to accelerate training.

---Original--- From: "lilslsls"<notifications@github.com> Date: Sat, May 23, 2020 13:54 PM To: "yjh0410/yolov2-yolov3_PyTorch"<yolov2-yolov3_PyTorch@noreply.github.com>; Cc: "Subscribed"<subscribed@noreply.github.com>; Subject: [yjh0410/yolov2-yolov3_PyTorch] Why is training so slow (#34)

4900 images and batchsize：64 gpu：2080TI

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

lilslsls commented 4 years ago

I used yolov2 training, not multi-scale, and a batch spent nearly one minute in training. An epoch has many batches. So I feel strange. Is it because the code parses XML.

yjh0410 commented 4 years ago

No multi scale？Aha！You can set num_workers as 8(default=8). 

Do you set --cuda to use GPU？ Its default value is False.

---Original--- From: "lilslsls"<notifications@github.com> Date: Sat, May 23, 2020 14:14 PM To: "yjh0410/yolov2-yolov3_PyTorch"<yolov2-yolov3_PyTorch@noreply.github.com>; Cc: "Comment"<comment@noreply.github.com>;"Jianhua Yang"<1394571815@qq.com>; Subject: Re: [yjh0410/yolov2-yolov3_PyTorch] Why is training so slow (#34)

I used yolov2 training, not multi-scale, and a batch spent nearly one minute in training. An epoch has many batches. So I feel strange. Is it because the code parses XML.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

lilslsls commented 4 years ago

Sorry, I didn't read the code carefully. Now it's fast to use CUDA. thank you

yjh0410 commented 4 years ago

You're welcome！！！(๑>؂<๑）

---Original--- From: "lilslsls"<notifications@github.com> Date: Sat, May 23, 2020 14:27 PM To: "yjh0410/yolov2-yolov3_PyTorch"<yolov2-yolov3_PyTorch@noreply.github.com>; Cc: "Comment"<comment@noreply.github.com>;"Jianhua Yang"<1394571815@qq.com>; Subject: Re: [yjh0410/yolov2-yolov3_PyTorch] Why is training so slow (#34)

Sorry, I didn't read the code carefully. Now it's fast to use CUDA. thank you

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

bruce1408 commented 4 years ago

SET CUDA TRUE To use GPU!

envs

use GPU

but when i set cuda True, the training speed up obivous! Epoch 1/250][Iter 10/517][lr 0.000000][Loss: obj 212.88 || cls 7.12 || bbox 9.07 || total 229.07 || size 416 || time: 2.20]

yjh0410 commented 4 years ago

SET CUDA TRUE To use GPU!

envs

  • 2080ti x 1
  • ubuntu 16.04
  • python 3.6
  • pytorch 1.5.0
  • cpu cores :8
  • cpu processor num :16

without GPU

when i use cpu to train the model ,the training is sooooooo slow, such as the blow info... [Epoch 1/250][Iter 0/517][lr 0.000000][Loss: obj 210.70 || cls 4.88 || bbox 8.88 || total 224.46 || size 416 || time: 47.35]

use GPU

but when i set cuda True, the training speed up obivous! Epoch 1/250][Iter 10/517][lr 0.000000][Loss: obj 212.88 || cls 7.12 || bbox 9.07 || total 229.07 || size 416 || time: 2.20]

Thanks for your advice！HAHA！

The reason why I didn't set cuda as True is that I wanna indirectly remind everyone who plans to try this project to use GPU to train the model. If no GPU，it's a little hard to study object detection.