Efficiently evaluate the network again on same minibatch

[Ark-kun]

Suppose, I have a big network net. I've just evaluated the loss function:

c.cross_entropy_with_softmax(net, labels).eval(data_map)

now I want to evaluate a slightly different loss function.

c.cross_entropy_with_softmax(net * 5, labels).eval(data_map)

Or maybe I've modified a constant that's only used near the end.

coeff.set_value([5])
c.cross_entropy_with_softmax(net * coeff, labels).eval(data_map)

What should I do to ensure the unchanged network part is not re-evaluated?

[eldakms]

I'm not aware of any functionality in CNTK to support evaluation of the same minibatch twice skipping some computations. This would require preserving the state of the previous minibatch and analysis what was changed in between.

Possible solutions that come to my mind (maybe there are more features that would allow you accomplish this, not sure, I'm not on CNTK team): 1) Combine several loss functions if you know all of them in advance. CNTK should be able to give outputs for all nodes you pass in Forward. 2) Control evaluation yourself: split the model in two parts: static and dynamic. Evaluate static part once, dynamic as many times as you want with whatever changes you want. should be easy enough for evaluation (not for training)

Thanks!

[Ark-kun]

@eldakms Unfortunately neither of the options are possible in my case. Imagine that I just want to do Forward+Backward+Update+Forward+Update+Forward+Update+Forward. I want all the weights to change, so I cannot segment the network or add another loss.