Cuda gpu

[mariembenslama]

Hello,

I have a RAM with 13GB, I activated cuda in the params.py file but the training and test are still slow comparing to the capacity of my machine.

It's supposed to run quickly I mean.

I'm wondering if cuda is actually working or not in reality?

[Holmeyoung]

@mariembenslama since our displayInterval is set to 100 and batchSize is set to 64, it means the training loss will be printed every 6400 samples. You can set the displayInterval to 10 then it may seems ten times quickly. Of course, you can use the command watch -n 1 nvidia-smi to checkout the using case of GPU, it will show something like

Sun Apr 28 17:44:06 2019
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 410.104      Driver Version: 410.104      CUDA Version: 10.0     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  Tesla P40           Off  | 00000000:3B:00.0 Off |                    0 |
| N/A   36C    P0   128W / 250W |   1063MiB / 22919MiB |     66%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0    249645      C   python                                      1053MiB |
+-----------------------------------------------------------------------------+

If there are still problems, please let me know, thank you~

[mariembenslama]

Thanks! I guess it's good now :) it jumped quickly to a good intervalle of accuracy and loss ^-^

Thank youuu~~~ Is the model (DL in general) always taking too long to grasp all the alphabets? I have about 3340 alphabets, at epoch 9/1000 Loss is 0.5456 and Accuracy is 0.004 but it's growing up. Right now I have 100000 train and 10000 test, I will train this sample, add a checkpoint, then add more data and train again.

Do you think by completing the train it'll give a good accuracy in the end?

[Holmeyoung]

@mariembenslama I believe you can do it! But you can add more training data to it to let it know the right decline direction. If you have more training data, after 9/1000 epoch, the loss can be lower! Good luck!

[mariembenslama]

Thank you! I will do as you say 😊

Thanks a lot again 😀😀😀