Increase Training Performance

[hakanonal]

First run executed ~200 episodes and after 4h, 1,2min per step was very slow. Try to decrease single episode execution time under 0,5min.

[hakanonal]

• After second run this issue is prioritized for me. Practically current performance is not usable at all.
• So simply I will increase the server's available CPU and see what is going to happen.
• a sweep has been restarted curentlly I am monitoring this run
• A new metric may be help to see if performance improvement solid. Avarage time of single episode.
• The new metric has been added I have stoped this run and started a new one
• Eventhough we have just added the metric and start to measure the performance and do not know if this new run is improved or not it is obvious that it is still slow.

[hakanonal]

• I have removed the old state read procedure. to improve performance. I read the first state before beginning the game, and after that I only read state for new_state. beforre going into next step I copy new_state to old_state to use it on next loop. By this way I have removed one read state action.
• I have realized that I am measuring the duration not only one action but one episode. I think it is wrong metric to improve performance since if the agent plays better, one episode will have more actions to play which will increase the duration. However this state will not show the program performance.
• I have decided to move duration metric to keep not whole episode but only keep one action.
• Well I am having trouble to understand the what duration metric means. duration itself show the last action loop duration however avg_duration shows the average of one whole episode's duration, which I find a little bit confusing.
• To increase performance one option is to change the model. So I have researched how some other people accomplish an agent to teach how to play flappy bird with a model that includes conv. layers. Here is one example. In this example he feeds the model with not only one state but stitches upto 4 states and feeds in grayscale.
• This example is particularly interesting..