Closed mbaroni closed 2 years ago
This would allow to perform a post-hoc language analysis decoupled from the training loop #150
Changes in #232 allow to load a pretrained game and perform an evaluation loop on any user-provided data, thus decoupling training and testing
Is your proposal related to a problem?
EGG seamlessly integrate validation, but there is no straightforward way to include a test set.
Describe the solution you'd like to have implemented
My favorite solution would be to have an easy way to save the trained agents, so that after training we can feed them arbitrary test sets.
Describe alternatives you've considered
Perhaps the simpler alternative would be to have a test option in the training, taking a further test set as option. However, it seems like a waste of resources to have to do a full re-training every time we change the test set.