Digits Details_validation vs test

[samar166]

Could you please tell me how I can get details of the accuracy (val) and loss(val) in Digits? What is difference between validation and test in Digits? more specifically in the attached picture and the diagram

Appreciate your help.

[gheinrich]

Hello, in Alexnet and most image classification networks, the loss function that we use is the logistic loss. See definition in Caffe: http://caffe.berkeleyvision.org/doxygen/classcaffe_1_1MultinomialLogisticLossLayer.html The optimization goal is to reduce this loss.

The accuracy is a metric that we track the percentage of correctly classified images. See: http://caffe.berkeleyvision.org/doxygen/classcaffe_1_1AccuracyLayer.html

For an overview of what the train, validation and test sets are: https://fr.coursera.org/learn/machine-learning/lecture/QGKbr/model-selection-and-train-validation-test-sets

[samar166]

Thank you so much for your response. I will go through them accurately, but just want to know if the validation is the inference as it is shown in the attached picture?

[gheinrich]

Yes, "validation" consists in performing inference on your model on all samples from the validation set. Since you know the ground truth for those samples you can compute quality metrics, like accuracy.

[samar166]

Thank you so much again. Is the validation accuracy being computed at each iteration? or each epoch? Also I don't know exactly how often the weights are being updated while training? Are they being updated at each epoch?

[gheinrich]

Those questions are better asked on the users list. The validation accuracy is computed on the full validation set at every snapshot interval epoch (configured in DIGITS UI, defaults to 1 and can be a fractional number like 0.5). For the record, one "epoch" is one pass through the full training set. Weights are updated on every iteration (i.e. every batch_size * batch_iteration samples on training set).

Closing.