search

CRBS / cdeep3m

Please go to https://github.com/CRBS/cdeep3m2 for most recent version
Other
58 stars 10 forks source link

How to obtain the training accuracy from the log file? #72

Closed cakuba closed 4 years ago

cakuba commented 5 years ago

Hi,

I'm using the script "PlotValidation.m" to obtain the figures on how the accuracy and loss change during the training stage. I notice that in the default output figure of"accuray.pdf" , there is no information on the training accruacy. But in the other figure of "loss.pdf", the curve of the traing loss is plotted. So I'm wondering if there is some way to recover the traning accuracy from the log file. Thanks for your help.

Brett

haberlmatt commented 5 years ago

Hi Brett, we aren't actually tracking the training accuracy since we track loss for both training and validation and accuracy for validation. PlotValidation.m runs a parsing on the log file you mentioned and stores the data in csv files for training and test column format for train_output csv (NumIters,Seconds,LearningRate,loss_deconv_all) column format for test_output csv (NumIters,Seconds,LearningRate,accuracy_conv,class_Acc,loss_deconv_all)

cakuba commented 5 years ago

Thanks for the super-fast response! So does the original log file ("caffe.bin.INFO") contain the training accuracy information? If so, I guess that we can revise this Matlab script to have it shown...

Thanks.

Brett

haberlmatt commented 5 years ago

Hi Brett,

No, you'd have to add a layer to measure training accuracy for 1fm 3fm and 5fm in each model folder, you can see it on the MNIST example of caffe https://github.com/BVLC/caffe/blob/master/examples/mnist/mnist_autoencoder_solver.prototxt and https://github.com/BVLC/caffe/blob/master/examples/mnist/mnist_autoencoder.prototxt line 18-25 similarly you would add it in the test phase in our train_val.prototxt you could use same as in lines 49-93 and use for the data_source "train_file.txt" (instead of "valid_file.txt") It might add some time, depending on the batch size you use. Once you added the layer it should be parsed automatically and the csv files will have that info. You'd just need to check which column to plot.

Best, Matt

cakuba commented 5 years ago

Hi, Matt,

Thanks for the detailed instruction. I will see if the layer can be added and the training accuracy was included in the log file. Will let you know.

Best, Brett