Got 65% ACC for BYOL on ImageNetdogs

[kasteric]

Hi, I am really sorry to bother u, but I only get 65% ACC for BYOL on ImageNetdogs, which is 4% lower than reported in paper, and I have made hard trials to figure out the reason, but still have no idea. Is there anything I miss? Thanks so much! Here is my config file:

batch_size: 64
num_devices: 4
momentum_base: 0.996
momentum_max: 1.0
momentum_increase: true
dataset: imagenetdogs
eval_metric:
  - nmi
  - acc
  - ari
whole_dataset: true
encoder_name: resnet34
epochs: 1000
feat_dim: 256
hidden_size: 4096
#img_size: 96
img_size: 96
lambda_predictor_lr: 10
learning_rate: 0.05
learning_eta_min: 0.
reassign: 1
save_freq: 100
save_checkpoints: true
shuffling_bn: true
symmetric: true
temperature: 0.5
#use_gaussian_blur: false
use_gaussian_blur: true
warmup_epochs: 50
weight_decay: 0.0005
dist: true
data_resample: true
v2: true
#byol_transform: true
test_resized_crop: true

model_name: propos
cluster_loss_weight: 0.
latent_std: 0.

Here is my performance curve:

[Hzzone]

Hi, could you turn on the flag use_copy? This flag uses the same model to extract the features during ddp training.

[kasteric]

Thanks a ton! I will try this.