search

keras-team / keras

Deep Learning for humans
http://keras.io/
Apache License 2.0
61.58k stars 19.42k forks source link

Optimizing prediction performance #3071

Closed Serhiy-Shekhovtsov closed 7 years ago

Serhiy-Shekhovtsov commented 8 years ago

I am working on reinforcement learning task and it requires calculating prediction too many times. I have found that 56,87% of cumulative time is taken by _predict_loop method. Also I have found that installing CUDA and enabling GPU calculation doesn't help. Is there any configuration tricks that can help here? If this is something that will require updating keras code - I am happy to help.

githubnemo commented 8 years ago

It seems that you use it wrong (according to the discussion on stackoverflow) and this is not an actual problem with Keras.

Serhiy-Shekhovtsov commented 8 years ago

What I have found so far is using batch_size when doing prediction can do a great help. So I will refactor my algorithm to make predictions in batches.

aravindr93 commented 8 years ago

I got only a partial picture from the discussion on stackoverflow. The biggest problem I see is the way in which the function approximation is set up. I believe you take both state and action as inputs, to approximate something of the form scalar = Q(s,a).

I think this is a bad idea. A better approach is: action = Q(state) i.e. have one unit at final layer for each action. If this isn't working for some reason, try: (a) A better exploration policy (eg pick action from soft-max distribution of outputs); (b) use actor critic architecture.

Serhiy-Shekhovtsov commented 8 years ago

I am not sure how softmax can help me, because I use prediction to choose action with biggest Q value. And if I am getting it right, argmax(x) is always same as argmax(softmax(x)). Also it's easy to train the model on pure calculated Q value. Regarding actor critic architecture - thank you for this suggestion, I am checking it. Btw, I have formulated my original problem on Cross Validated. This huge number of predictions I am doing is just a way to workardound the original issue.

Meanwhile I have redesigned my code to run predictions in butches. Now it runs much faster.

denlecoeuche commented 8 years ago

If you want to take advantage of your GPU to speed up the calculation you should take a look at the predict_generator method (as well as fit_generator for training).

Note that you will have to provide thread-safe generators, see #1638.