Train,Test and Validation division of data

[ashesh-0]

Hi, I'm not able to understand how the division of data was done for 3 fold evaluation in your paper. on RT-Gene dataset. Let me elaborate.

I'm clear on the division of persons for training data and validation data. Validation data is always [s014,s015,s016]. Training data becomes any two of the three.

1. 's001', 's002', 's008', 's010'
2. 's003', 's004', 's007', 's009'
3. 's005', 's006', 's011', 's012', 's013'

Issue

1. For test data, I see conflicting descriptions. At location of dataset link, it says that from the above mentioned 3 divisions, you pick 2 divisions for training and one for testing. There is no 's000' here. However, in your pytorch version of the code https://github.com/Tobias-Fischer/rt_gene/blob/master/rt_gene_model_training/pytorch/train_model.py#L180, I see 's000' included in test data in all three folds.
2. In the tensorflow implementation, https://github.com/Tobias-Fischer/rt_gene/blob/master/rt_gene_model_training/tensorflow/prepare_dataset.m#L137 the code divides the data for each subject into train and test.

I'm interested in knowing the configuration of 3 fold evaluatioin with which you have gotten the 3D gaze estimation results in your paper on RT-Gene dataset. Thanks !! :)

[Tobias-Fischer]

Hi @ashesh-0, Thanks for your inquiry!

• Indeed there was a mistake in the PyTorch version of our code - s000 should not be in the test data, but in the validation data instead. I fixed that in https://github.com/Tobias-Fischer/rt_gene/commit/900c78b11e5da28e4d3cf5745bd5d477aca90f63
• The split into train and test subjects in the Tensorflow version is here: https://github.com/Tobias-Fischer/rt_gene/blob/master/rt_gene_model_training/tensorflow/train_model.py - you can see that in this version of the code the validation data stems from a subset of the subjects' data (https://github.com/Tobias-Fischer/rt_gene/blob/900c78b11e5da28e4d3cf5745bd5d477aca90f63/rt_gene_model_training/tensorflow/train_tools.py#L72). In practice, we found that there is no significant difference between using other subjects for validation or a subset of the data from the same subjects. Also, in Tensorflow we do not make use of subject 000 at all.

I would recommend using the PyTorch version (now with the fix).

I hope this helps - feel free to re-open this issue if you have more questions.

/cc @ahmed-alhindawi

[ashesh-0]

Thanks for the prompt response !