Support to perform test set validation during the training evolution

[peiyangL]

The acc and loss reported during the training process in trainer.py seems to be the training data metrics. And is there any easy-to-implement options to calculate the testing set acc during the training evolution in the current framework?

thank you~

[Renovamen]

Hi, none of the datasets I used provides a validation set, so I'm not able to obtain validation acc and loss during training.

And is there any easy-to-implement options to calculate the testing set acc during the training evolution in the current framework?

I don't think accessing test set during training stage is reasonable. But if you still want to do that, I guess you can simpily copy the test function in test.py to trainer/trainer.py, then modify the following code

https://github.com/Renovamen/Text-Classification/blob/4a4aa4001c402ed4371ebaabe1393b27794e5992/trainer/trainer.py#L205-L233

to

def run_train(self):
    start = time.time()

    # epochs
    for epoch in range(self.start_epoch, self.num_epochs):
        # trian an epoch
        self.train(epoch=epoch)

        # time per epoch
        epoch_time = time.time() - start
        print('Epoch: [{0}] finished, time consumed: {epoch_time:.3f}'.format(epoch, epoch_time=epoch_time))

        self.test()

        # some other code...