search

XtractOpen / Meganet.jl

A fresh approach to deep learning written in Julia
http://www.xtract.ai/
Other
14 stars 9 forks source link

Pull request/9de97af9 #38

Closed jgranek closed 6 years ago

jgranek commented 6 years ago

@eldadHaber @lruthotto What is Q? It seems like its a projection matrix of some kind..? It can be either an operator (Identity) or a 2D Array?

Also - Jmv doesn't seem to get used anywhere in the CIFAR10 example...?

coveralls commented 6 years ago

Coverage Status

Coverage increased (+0.2%) to 78.288% when pulling 9de97af972dff42a272bf1859e36dc5af8c38eba on jgranek:pull-request/9de97af9 into 1988cea500b0164d0d15c0932463cb5fa83070dd on XtractOpen:dev.

lruthotto commented 6 years ago

Q is a projector that takes features from intermediate time steps and forwards them to the classifier. It can be any linear operator.

It might well be that Jmv is not being used in the CIFAR example. First, note that Jmv is the Jacobian and not its transpose (used in backprop).In first order methods like SGD you wouldn't use Jacobians much. However, they'll be used in second-order methods and also to test derivatives.