Closed amsword closed 4 years ago
Hi, pre-training is not supported by this code release. In the paper we describe very precisely how to do it, so you could actually try it out by adding support for ImageNet-21k dataset, weight-decay, and few other missing things.
But we do not plan to actively support the pre-training part in this code-base.
@lucasb-eyer it is based on supervised training with cross entropy, or based on self-supervised training (e.g. simCLR from Hinton)?
Since there were more concrete questions in #26, I gave some more information there, including on the loss. Hence, I'm closing this issue.
From the paper and the README, i cannot find any hint about how the pre-training runs. Are there any document on this?
Thanks