Get more output during training

[claell]

I'd like to get some more output like training accuracy and test loss. Is this easily achievable?

[ranahanocka]

Hi @claell ,

I suppose it depends a bit on your problem. You can try to play with the network hyperparameters: more convolution layers, more or less skip connections, more or less pooling, learning rate, etc.

You can also try to do more data-augmentation. By that I mean not just the options we have in the code, but also re-meshing the objects you have. One way to do that is performing edge flips (with our code), then you can simplify with the blender script. The resulting meshes will be different for the same starting mesh.

Do you have high training accuracy and low test accuracy?

[claell]

I guess you have misunderstood the question. I am trying to get a more verbose output. Currently the training accuracy and test loss is not printed and I'd like to also see those values.

But apart from that, yes, I probably have rather high training accuracy while the test accuracy stays rather low (although I cannot currently check that without more verbose output). I guess that is a sign for overfitting? Also the classification problem I want to solve is rather complicated, meaning not just object classification, but learning a more complicated kind of classification.

[ranahanocka]

Hi @claell ,

The code indeed currently prints the training loss and testing accuracy (and also saves it to a log file).

Yes, high training accuracy and low test accuracy is a good indication of overfitting. For overfitting problems, see my suggestion above about data augmentation.

[claell]

Hi @ranahanocka,

I wrote

Currently the training accuracy and test loss is not printed

You replied

currently prints the training loss and testing accuracy

Those don't contradict each other, I think we both are correct with our statements. However, I'd also like to print those other stats, so the values for each accuracy and loss can be compared between train and test indicating how much they differ from each other.

Regarding overfitting: This might not be related to the network architecture but to the task I try to solve which is rather complicated and might not be suited that well for usage of ANNs in general (I am also testing different network approaches). One thing I will try before is breaking the task down into smaller, hopefully easier to solve classification problems. I will keep your suggestion in mind, though!

[claell]

Apparently the values are also not available in the Tensorboard data. Just training accuracy and test loss.

[ranahanocka]

Hi @claell --

Currently the training accuracy and test loss is not printed

I see. I did not add this functionality... you can modify the code to print these if you'd like -- to print training accuracy: you need to run model.test() on the training data (as I did in the test.py script). Alternatively, you can manually check this by making a dummy dataset folder with the "training" files in the test folder.

To print the test loss, you should run the loss_criteria on the output -- without running the optimizer. i.e., maybe you can just modify the model.test method, so it will calculate and return the loss: loss = self.criterion(out, self.labels)

[claell]

Alright. I will look into the code and see if I can get it to work. Will the ways you proposed also add the results to the Tensorboard data?

[ranahanocka]

To add plots to the tensorboard you should add another method in the Writer Class. Or modify the existing one to take the plot name as an argument.

For example: this is the method that plots the loss. You will need to change the name of the plot (which for this example is data/train_loss)