search

FrancoisLasson / Temporal_DBN

A Temporal Deep Belief Network implementation using Theano
2 stars 1 forks source link

Memory actualization frequency #14

Closed FrancoisLasson closed 8 years ago

FrancoisLasson commented 8 years ago

At the moment, we actualize memory (past visible) at each frames. So, a large number of past visible layers are required to have a long temporal influence and a large number of past visibles means a huge network for which the training phase will be long and difficult.

An idea could be to actualize past visible each N frames (N∈ℕ is a new parameters), In other words, have a high temporal influence without increase the size of our network.

Have to test!

FrancoisLasson commented 8 years ago

Yesterday, I implemented this new function. (Sigmoid error in LogReg has already been corrected) Results are really encouraging!

Without frequency functionality : per_crbm_1461055163 11expe_number_0

with frequency functionality: (validation PER=24,15% ; test PER=24,5%) per_crbm_1461123172 73 (Here, we actualize with N=6 in other words, 1 frame each 6 frames. Or, in this database fps=120, so we actualize each 6/120=1/20=0,05 seconds. In these experiments, network has 10 past visible layers, so temporal windows has an influence on 0,5 seconds