anandthakker / doiuse

:bomb: Lint CSS for browser support against caniuse database.
MIT License
1.24k stars 51 forks source link

Segnet multi-gpu #52

Closed Maxfashko closed 8 years ago

Maxfashko commented 8 years ago

Hello! I am sorry that I am not writing on the topic. I sent you a message via e-mail, but you have not answered me.

I saw that you had success in segnet work using several gpu. I went to https://github.com/developmentseed/caffe/tree/segnet-multi-gp and made this assembly (cafe). I used three out of four tesla m2090 (5gb ram). But I didn't see any performance improvement.

| NVIDIA-SMI 352.39 Driver Version: 352.39 |
|-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===============================+======================+======================| | 0 Tesla M2090 Off | 0000:19:00.0 Off | 0 | | N/A 59C P12 30W / 225W | 10MiB / 5375MiB | 0% Default | +-------------------------------+----------------------+----------------------+ | 1 Tesla M2090 Off | 0000:1A:00.0 Off | 0 | | N/A 59C P1 139W / 225W | 4812MiB / 5375MiB | 87% Default | +-------------------------------+----------------------+----------------------+ | 2 Tesla M2090 Off | 0000:1E:00.0 Off | 0 | | N/A 59C P0 159W / 225W | 4457MiB / 5375MiB | 19% Default | +-------------------------------+----------------------+----------------------+ | 3 Tesla M2090 Off | 0000:1F:00.0 Off | 0 | | N/A 59C P0 141W / 225W | 4457MiB / 5375MiB | 0% Default | +-------------------------------+----------------------+----------------------+

I used three gpu. But the learning rate didn't change. 20 iterations took over 40 seconds, as if I use one gpu.

I tried to change batch size in segnet_train.prototxt to value 2 or 3 (on m2090 segnet works only if batch size is 1, because I have 5 gb ram only). But I received the message "Out of memory". Please tell me if I can increase the learning rate if I use two or more gpu. If so, what should I do? Can I increase the value of upsample layer in segnet_train.prototxt?

layer { name: "upsample5" type: "Upsample" bottom: "pool5" top: "pool5_D" bottom: "pool5_mask" upsample_param { scale: 2 upsample_w: 30 upsample_h: 23 } }

I wan to change the values to upsample_w : 45, upsample_h : 45 in order to enable neural network to accept images 720*720 as an input parameter. Could it be possible using two or more gpu?

Thank you in advance.

anandthakker commented 8 years ago

This is the wrong place for this question. Please redirect your question to the caffe-users forum

Maxfashko commented 8 years ago

.....help from you will not wait.