search

google-research / big_transfer

Official repository for the "Big Transfer (BiT): General Visual Representation Learning" paper.
https://arxiv.org/abs/1912.11370
Apache License 2.0
1.5k stars 175 forks source link

no document about how to do pre-training #2

Closed amsword closed 4 years ago

amsword commented 4 years ago

From the paper and the README, i cannot find any hint about how the pre-training runs. Are there any document on this?

Thanks

lucasb-eyer commented 4 years ago

Hi, pre-training is not supported by this code release. In the paper we describe very precisely how to do it, so you could actually try it out by adding support for ImageNet-21k dataset, weight-decay, and few other missing things.

But we do not plan to actively support the pre-training part in this code-base.

amsword commented 4 years ago

@lucasb-eyer it is based on supervised training with cross entropy, or based on self-supervised training (e.g. simCLR from Hinton)?

lucasb-eyer commented 4 years ago

Since there were more concrete questions in #26, I gave some more information there, including on the loss. Hence, I'm closing this issue.