search

peteanderson80 / Matterport3DSimulator

AI Research Platform for Reinforcement Learning from Real Panoramic Images.
Other
503 stars 130 forks source link

Teacher-forcing performance in test unseen #23

Closed khanhptnk closed 6 years ago

khanhptnk commented 6 years ago

Why didn't the paper report teacher-forcing performance in test unseen? (table 1).

peteanderson80 commented 6 years ago

We only wanted to run one model on the test set. We used the validation set to determine the best training scheme (out of the options we tried), so we didn't need the test set to also confirm this. Over time, performing too many runs on the test set leaks information and leads to overfitting.