Seeking Guidance on Reproducing Reported Accuracy

[wolfrosnet]

I am currently working on reproducing the reported accuracy of Protonet on miniImageNet with a 1-shot scenario (63.17% -> 66.63%). I initially trained the classifier-baseline using the provided code and attempted to train ProtoDiff.

However, I encountered challenges as the accuracy scores were not as expected. Specifically, after training both the classifier-baseline and ProtoDiff on convnet4 and resnet12, the accuracy scores were suboptimal (resnet12: 63.25% -> 62.88%, convnet4: 41.61% -> 31.52%).

Do you have plans to release the final code?

I would appreciate any guidance or insights you can provide on the configurations I used. Please find the details below:

[train_classifier_mini.yaml] train_dataset: mini-imagenet train_dataset_args: {split: train, augment: resize} val_dataset: mini-imagenet val_dataset_args: {split: train_phase_val} fs_dataset: mini-imagenet fs_dataset_args: {split: test} eval_fs_epoch: 5

model: classifier model_args: encoder: resnet12 encoder_args: {} classifier: linear-classifier classifier_args: {n_classes: 64}

batch_size: 256 max_epoch: 150 optimizer: sgd optimizer_args: {lr: 0.1, weight_decay: 5.e-4, milestones: [90]}

save_epoch: 5 visualize_datasets: true

[train_meta_mini.yaml] train_dataset: mini-imagenet train_dataset_args: { split: train } tval_dataset: mini-imagenet tval_dataset_args: { split: test } val_dataset: mini-imagenet val_dataset_args: { split: val }

model: meta-baseline model_args: encoder: resnet12 encoder_args: { } load_encoder: ./save/classifier_mini-imagenet_resnet12/epoch-last.pth n_way: 5 n_shot: 1 n_query: 15 train_batches: 200 ep_per_batch: 4

max_epoch: 100 optimizer: sgd diffusion_optimizer: sgd optimizer_args: { lr: 1.e-4, weight_decay: 5.e-4} diffusion_optimizer_args: { lr: 1.e-3, weight_decay: 5.e-4, milestones: [30, 50, 80]}

visualize_datasets: True overfit_lr: 0.000001 overfit_iterations: 10 loss_coff: 10.

save_epoch: 20

transformer: ema: true predict_xstart: true max_freq_log2: 14 num_frequencies: 128 n_embd: 512 encoder_depth: 1 decoder_depth: 1 n_layer: 12 n_head: 16 dropout_prob: 0.1 len_input: 3 numstep: 10 time: 100

[test_few_shot.yaml] dataset: mini-imagenet dataset_args: {split: test}

load: ./save/meta_mini-imagenet-1shot_meta-baseline-resnet12-1e-06-10/max-va.pth

transformer: ema: true predict_xstart: true max_freq_log2: 14 num_frequencies: 128 n_embd: 512 encoder_depth: 1 decoder_depth: 1 n_layer: 12 n_head: 16 dropout_prob: 0.1 len_input: 3 numstep: 1 time: 100

visualize_datasets: True overfit_lr: 0.000001 overfit_iterations: 10 loss_coff: 10.

[wangpeng0702]

Hello, did you obtain the 63.17% metric in test_few_stot.cy or train_1. py？