search

google-research / mixmatch

Apache License 2.0
1.13k stars 163 forks source link

Number of validation samples #20

Closed hankook closed 4 years ago

hankook commented 4 years ago

I found that scripts in the runs/ directory use only one sample for validation, e.g., here.

The paper also said that the median error rate of the last 20 checkpoints is used when reporting, which means that no validation is required. Is this the reason why the code uses only one validation sample?

Am I correct? If not, how many validation samples are used?

david-berthelot commented 4 years ago

Yes that is correct, we used 1 because the code doesn't allow for 0. Also we didn't really tune per dataset, we used almost the same parameters for everything.

hankook commented 4 years ago

Thanks for your reply and this public repo :)