EderSantana
commented
8 years ago Hi @zoli333 tkx for your interest in Seya. Please consider making a pull request with your modifications to the keras1 branch.
I think that I interchanged the functions because I named the behavior in the wrong way. But please, if you debug that let me know.
Hi
I want to update your WinnerTakeAll layer to work with the new keras. So I implemented the Spatial + lifetime algorithm based on this article with the help of your code. It works in the new keras version but I dont know if this is correct: http://arxiv.org/pdf/1409.2752v2.pdf
Here is my code (its theoretically compatible with keras 1.0.7, so with the newest version) wta.zip
Basically my batch size is 100, and the filters are 64 size. For example in a 100x64x5x5 tensor, there will be 100 maximum values, because the maximums are computed across 1,2,3 dimension axes (as the article mentioned). Then I drop out "k" of them, in this case from 100 batch and I drop 20% of them out, there will remain just 80 out of 100. This was basically in the article when the Spatial+Lifetime winner-take-all was applied.
My question is the winnertakeall function correct? In the feed-forward case is this function returns the masked x (the k% of the maximums are zeroed out), but more importantly is it return the masked filters in the backward case? So the derivatives are computed through the non-zero values? If not how can I set this. Should I use masking here or not, because I don't know how to do this
Another question, Why the spatial and lefetime functions are interchanged? When one call the function spatial=5, lifetime=5 these conditions will be applied: if self.n_largest: Y = self.wta_largest(Y, self.n_largest) if self.wta_spatial: Y = self.wta_lifetime(Y, self.spatial) if self.wta_lifetime: Y = self.wta_spatial(Y, self.lifetime) So if I call the both the spatial and lifetime (these two parameters are set) and set the previous_mode to False, First I call the spatial, but this calls the lifetime function with the spatial parameter? And then if I set the lifetime parameter too (which is not set to zero its 5) then the code calls the wta_spatial function, with the lifetime parameter ? I don't understand this. Could you explain it why?
Thank you