validation data is not true validation data

[harbecke]

Since validation data is generated every epoch for the training set, it can be seen by the model in previous iterations. Furthermore positions from the same game are very related. We should find a solution to select only a subset of positions from one game or create games only for validation to make sure the validation loss is different from the training loss.

[harbecke]

See for example with the config config.txt

The puzzle loss increases which strongly points to overfitting, whereas the validation loss still goes in the direction of the training loss.

[cleeff]

Yes, I'm sure it is currently strongly correlated to training loss. I think drawing samples from different games for training and validation is the only way to go here, but this will require some additional work. Maybe the easiest / safest way is to have separate files for training and validation data.

[harbecke]

Yeah that sounds good to me. Just unassign the person you don't want to implement this ;)

[cleeff]

Working on MCTS ;-)

[harbecke]

solved with 9fa8cf9fe18b33472430fe6bd52e61f1f827367f