eval and run at same time?

[jonathanstrong]

Is it possible to execute the updates from an optimizer (run) and return the loss at the same time (eval)? I can't figure out how to do this. The theano.function interface I'm familiar with accepts return parameters and a separate list/dict of updates. Perhaps with autograd the eval has to be in updates, but I don't know how to retrieve the value.

[raskr]

Yes, you can do it (sorry for poor documentation). eval and run are almost same; the slight difference is in postprocessing:

• eval returns Vec<Result>
• run unwraps the vec automatically, and return nothing.

So pseudo code would be like:

let loss = ...
let mut update_ops: Vec<_> = adam.compute_updates(params, grads);
update_ops.insert(0, loss);  // update_ops: [loss, update_of_w, update_of_b]
// Evaluate those three nodes at the same time.
// `run` can always be replaced with `eval`.
let ret = ag::eval(&update_ops, &[]);
println!(loss = {:?}, ret[0]);
assert_eq!(ret[1], ag::runtime::EvaluationError::NoOutput);
assert_eq!(ret[2], ag::runtime::EvaluationError::NoOutput);

Note1: ag::eval is smart enough to avoid duplicated computation in update_ops and loss. (e.g. affine, addition of bias, cross-entropy don't run twice)

Note2: If you want see the current parameters, it is just in Tensor::persistent_array