Open kuroyakov opened 7 years ago
Your problem seems to be caused by automatic broadcasting performed by the addition. The vector of size [nHidden] is automatically broadcast along the first dimension of the matrix of dimension [nHidden;nBatch] resulting in a [nHidden;nHidden] matrix.
You sould be able to solve the problem by manually specifying the dimension along which you want to broadcast the vector. Just replace
let hiddenAct = hiddenWeights .* input.T + hiddenBias
with
let hiddenAct = hiddenWeights .* input.T + (Expr.reshape [nHidden;SizeSpec.broadcastable] hiddenBias)
This way a broadcasting dimension is added as the second dimension and the bias vector should now be broadcast to a matrix of the required shape.
Let me know if this worked or if you have further questions.
Thanks so much for your instruction. My code worked fine!
Unfortunately, When I proceed training section, test loss was same value each iterations. I revised my code in the last few lines as follows:
// Training
let opt = Optimizers.GradientDescent (loss, mi.ParameterVector, DevHost)
let optFn = mi.Func opt.Minimize |> opt.Use |> arg2 input target
// Set learning rate
let optCfg = {Optimizers.GradientDescent.Cfg.Step=1e-1f}
for itr = 0 to 1000 do
let t = optFn mnist.Trn.All.Input mnist.Trn.All.Target optCfg
if itr % 50 = 0 then
let l = lossFn mnist.Tst.All.Input mnist.Tst.All.Target |> Tensor.value
printfn "Test loss after %5d iterations: %.4f" itr l
I got the result:
Test loss after 0 iterations: 2.3019
Test loss after 50 iterations: 2.3019
...
Test loss after 950 iterations: 2.3019
Test loss after 1000 iterations: 2.3019
Please let me know what's wrong about my code.
BTW Do I need to create another issue for the further question?
Hey kuroyakov,
there has been a lot of work going on in the project and unfortunately the docs and code example are a little bit behind right now. I will look over them in the next few days and come back to you as soon as possible.
Sebastian
Hi surban.
Thank you for your response and I understand and am proud that you're doing. I'm looking forward to your update.
I examined DeepNet sample code referenced by (http://www.deepml.net/model.html)
However, I executed my code and got System.Exception. Additional information was as follows:
The exception occurred at
let hiddenAct = hiddenWeights .* input.T + hiddenBias
I expecthiddenBias
will be[nHidden; nBatch]
shapes, but[nHidden; nHidden]
.My complete code is as follows:
My environment are as follows:
I'm sorry if I'm misunderstanding about your sophisticated library. Could you please let me know how to fix this problem?