Could you provide the pre-trained models for ProPos on ImageNet-1k, ImageNet-50/100/200

[XY-ATOE]

Thanks

[Hzzone]

The checkpoint on ImageNet-1k is available at https://drive.google.com/drive/folders/1X45CQxiNFM1tW56sYWxfUyux7Nsbrs8F

I have not saved the checkpoints on the subset of ImageNet.

[XY-ATOE]

What about the config on the subset of ImageNet? This is my config on ImageNet-50. Is it ok？Could you give me some advice?

batch_size: 64
num_devices: 4
momentum_base: 0.996
momentum_max: 1.0
momentum_increase: true
dataset: imagenet_50
eval_metric:
  - nmi
  - acc
  - ari
whole_dataset: true
encoder_name: resnet50
epochs: 300
feat_dim: 256
hidden_size: 4096
#img_size: 96
img_size: 224
lambda_predictor_lr: 10
learning_rate: 0.05
learning_eta_min: 0.
reassign: 50
save_freq: 50
save_checkpoints: true
shuffling_bn: true
symmetric: true
temperature: 0.5
# use_gaussian_blur: false
use_gaussian_blur: true
warmup_epochs: 50
weight_decay: 0.0005
dist: true
data_resample: true
v2: true
#byol_transform: true
test_resized_crop: true

model_name: propos
cluster_loss_weight: 0.1
latent_std: 0.001

wandb: false
project_name: 'clustering_archives'
entity: 'zzhuang'

[XY-ATOE]

NMI of previous config reached 0.50. This is another config I modified from the config of imagenet-1k. NMI of this one only reached 0.30.

batch_size: 32
# num_workers: 16
num_devices: 4
momentum_base: 0.99
momentum_max: 1.0
momentum_increase: true
dataset: imagenet_50
eval_metric:
  - nmi
  - acc
  - ari
whole_dataset: true
encoder_name: resnet50
epochs: 300
feat_dim: 256
hidden_size: 4096
#img_size: 96
img_size: 224
lambda_predictor_lr: 1.0
learning_rate: 0.3
learning_eta_min: 0.
reassign: 50
save_freq: 50
save_checkpoints: true
shuffling_bn: true
symmetric: true
temperature: 0.5
# use_gaussian_blur: false
use_gaussian_blur: true
warmup_epochs: 10
weight_decay: 0.000001
dist: true
data_resample: true
v2: true
#byol_transform: true
test_resized_crop: true
# acc_grd_step: 16
model_name: propos
cluster_loss_weight: 0.001
latent_std: 0.0001
queue_size: 10240
exclude_bias_and_bn: true
lars: true
wandb: false
project_name: 'clustering_archives'
entity: 'zzhuang'

[Hzzone]

Sorry for the late reply. The results in the paper are trained with config https://github.com/Hzzone/ProPos/blob/master/config/cifar20_r34_propos.yml

You need to change the encoder to resnet50, and dataset to imagenet50/100/200. You can make the subset of imagenet by creating a soft link to each image folder. Credit to chatgpt:

import os

# Set the root path of the ImageNet dataset
imagenet_root = '/path/to/imagenet/dataset/'  # Replace with your actual path

# Specify the list of category names for which you want to create symbolic links
category_names = ['n04380533', 'n02123597', 'n02666196']  # Replace with your category name list

# Specify the target directory for the symbolic links
output_dir = '/path/to/output/directory/'  # Replace with the directory where you want to store the symbolic links

# Check if the output directory exists; create it if it doesn't
if not os.path.exists(output_dir):
    os.makedirs(output_dir)

# Iterate through the list of category names and create a symbolic link for each
for category_name in category_names:
    # Build the full path to the category directory
    category_dir = os.path.join(imagenet_root, category_name)

    # Check if the category directory exists
    if os.path.exists(category_dir):
        # Construct the target path for the symbolic link
        symlink_path = os.path.join(output_dir, category_name)

        # Create the symbolic link
        os.symlink(category_dir, symlink_path)
        print(f'Symbolic link created at: {symlink_path}')
    else:
        print(f'Category directory {category_name} does not exist in the ImageNet dataset')

[XY-ATOE]

My first config change encoder to resnet50, img_size to 224 and epochs to 300 ,as paper mentioned. But NMI of this config only reached 0.50.

[XY-ATOE]

I will try again with the following config. Waiting for my success!

batch_size: 64
num_devices: 4
momentum_base: 0.996
momentum_max: 1.0
momentum_increase: true
dataset: imagenet_50
eval_metric:
  - nmi
  - acc
  - ari
whole_dataset: true
encoder_name: resnet50
epochs: 300
feat_dim: 256
hidden_size: 4096
img_size: 224
lambda_predictor_lr: 10
learning_rate: 0.05
learning_eta_min: 0.
reassign: 50
save_freq: 50
save_checkpoints: true
shuffling_bn: true
symmetric: true
temperature: 0.5
use_gaussian_blur: false
warmup_epochs: 50
weight_decay: 0.0005
dist: true
data_resample: true
v2: true
use_copy: true

model_name: propos_cifar20_r34
cluster_loss_weight: 0.1
latent_std: 0.001

wandb: true
project_name: 'clustering_archives'
entity: 'zzhuang'

[Hzzone]

The config:

{
    "amp": {
        "desc": null,
        "value": false
    },
    "lars": {
        "desc": null,
        "value": false
    },
    "seed": {
        "desc": null,
        "value": 0
    },
    "faiss": {
        "desc": null,
        "value": false
    },
    "_wandb": {
        "desc": null,
        "value": {
            "t": {
                "1": [
                    1,
                    5
                ],
                "4": "3.7.2",
                "5": "0.10.31",
                "8": [
                    5
                ]
            },
            "framework": "torch",
            "cli_version": "0.10.31",
            "is_jupyter_run": false,
            "python_version": "3.7.2",
            "is_kaggle_kernel": false
        }
    },
    "epochs": {
        "desc": null,
        "value": 300
    },
    "syncbn": {
        "desc": null,
        "value": false
    },
    "dataset": {
        "desc": null,
        "value": "imagenet_50"
    },
    "feat_dim": {
        "desc": null,
        "value": 256
    },
    "img_size": {
        "desc": null,
        "value": 224
    },
    "momentum": {
        "desc": null,
        "value": 0.9
    },
    "reassign": {
        "desc": null,
        "value": 1
    },
    "run_name": {
        "desc": null,
        "value": "imagenet50_resnet50_w01_s0001"
    },
    "use_copy": {
        "desc": null,
        "value": false
    },
    "save_freq": {
        "desc": null,
        "value": 1
    },
    "symmetric": {
        "desc": null,
        "value": true
    },
    "batch_size": {
        "desc": null,
        "value": 64
    },
    "latent_std": {
        "desc": null,
        "value": 0.001
    },
    "local_rank": {
        "desc": null,
        "value": 0
    },
    "model_name": {
        "desc": null,
        "value": "offline_contrast"
    },
    "print_freq": {
        "desc": null,
        "value": 10
    },
    "data_folder": {
        "desc": null,
        "value": "/home/zzhuang/DATASET/scan_imagenet_subsets"
    },
    "hidden_size": {
        "desc": null,
        "value": 4096
    },
    "num_cluster": {
        "desc": null,
        "value": null
    },
    "num_workers": {
        "desc": null,
        "value": 32
    },
    "temperature": {
        "desc": null,
        "value": 0.5
    },
    "acc_grd_step": {
        "desc": null,
        "value": 1
    },
    "encoder_name": {
        "desc": null,
        "value": "resnet50"
    },
    "resume_epoch": {
        "desc": null,
        "value": 200
    },
    "shuffling_bn": {
        "desc": null,
        "value": true
    },
    "weight_decay": {
        "desc": null,
        "value": 0.0005
    },
    "byol_momentum": {
        "desc": null,
        "value": 0.996
    },
    "data_resample": {
        "desc": null,
        "value": false
    },
    "learning_rate": {
        "desc": null,
        "value": 0.05
    },
    "warmup_epochs": {
        "desc": null,
        "value": 20
    },
    "whole_dataset": {
        "desc": null,
        "value": false
    },
    "byol_transform": {
        "desc": null,
        "value": true
    },
    "fix_predictor_lr": {
        "desc": null,
        "value": false
    },
    "momentum_increase": {
        "desc": null,
        "value": false
    },
    "use_gaussian_blur": {
        "desc": null,
        "value": false
    },
    "cluster_loss_weight": {
        "desc": null,
        "value": 0.1
    },
    "exclude_bias_and_bn": {
        "desc": null,
        "value": false
    },
    "lambda_predictor_lr": {
        "desc": null,
        "value": 10
    }
}

The curve of NMI:

The log file has expired and been deleted by WANDB.

If I remember correctly, the results are reported on the test set of imagenet, eg, 50 classes, following SCAN.

[XY-ATOE]

Did you use pre-trained model ProPos on imagenet-50?

[Hzzone]

No

[XY-ATOE]

Thank you

[XY-ATOE]

I think test set you say refers to val set. The NMI of this config https://github.com/Hzzone/ProPos/issues/17#issuecomment-1772094172 is 0.50. I noticed that NMI is already greater than 0.20 at 0 epoch in your result. But my NMI at 0 epoch is only 0.15. I can‘t understand why it is happening.

[Hzzone]

I am sorry for the misleading. Actually, the curve of NMI did not start from epoch 0.

Current codebase does not report the clustering results (only knn ACC) on the test set. You can add the lines:

psedo_labels, cluster_centers = self.clustering(test_features, self.num_cluster)
results = torch_clustering.evaluate_clustering(test_features.cpu().numpy(),
                                                       psedo_labels.cpu().numpy(),
                                                       eval_metric=opt.eval_metric,
                                                       phase='test')
self.logger.msg(results, n_iter)

to https://github.com/Hzzone/ProPos/blob/cc35d8ca276aab8aa5dfb638fa15f919cf483824/models/basic_template.py#L444

Then, you can take a try the config below.

batch_size: 64
num_devices: 4
momentum_base: 0.996
momentum_max: 1.0
momentum_increase: true
dataset: imagenet_50
eval_metric:
  - nmi
  - acc
  - ari
# whole_dataset: true
encoder_name: resnet50
epochs: 300
feat_dim: 256
hidden_size: 4096
img_size: 224
lambda_predictor_lr: 10
learning_rate: 0.05
learning_eta_min: 0.
# reassign: 50
save_freq: 50
save_checkpoints: true
shuffling_bn: true
symmetric: true
temperature: 0.5
# use_gaussian_blur: false
# warmup_epochs: 50
weight_decay: 0.0005
dist: true
data_resample: true
v2: true
use_copy: true

warmup_epochs: 20
use_gaussian_blur: true # equal to byol_transform true
reassign: 1
whole_dataset: false # whole_dataset=true uses both train and val set
exclude_bias_and_bn: false
test_resized_crop: true # imagenet center crop during testing
test_freq: 1

model_name: propos_cifar20_r34
cluster_loss_weight: 0.1
latent_std: 0.001

wandb: true
project_name: 'clustering_archives'
entity: 'zzhuang'

[XY-ATOE]

I tested with the config you provided. I got NMI 0.51216, ARI 0.15267, ACC 0.23680. The result after initialization is NMI 0.15627, ARI 0.00395, ACC 0.08080. I noticed that your ARI is much higher than mine. This is the image distribution of my train set and val set. Is this distribution consistent with yours?

tensor([1300, 1300, 1198, 1300, 1300, 1300, 1240, 1301, 1300, 1300, 1300, 1300,
        1300, 1299, 1300, 1300, 1300, 1300, 1300, 1300, 1198, 1300, 1300, 1287,
        1300, 1270, 1288, 1300, 1300, 1300, 1300, 1264, 1300, 1300, 1251, 1263,
        1300, 1263, 1300, 1197, 1300, 1287, 1270, 1300, 1300, 1300, 1198, 1300,
        1300, 1300]))

tensor([49, 48, 50, 50, 49, 52, 51, 52, 50, 50, 51, 51, 51, 51, 51, 51, 51, 51,
        48, 49, 50, 51, 50, 50, 52, 49, 50, 51, 50, 50, 49, 49, 49, 50, 50, 49,
        50, 50, 49, 50, 50, 50, 50, 49, 49, 50, 50, 50, 48, 50])

[Hzzone]

I used the same classes from https://github.com/wvangansbeke/Unsupervised-Classification/tree/master/data/imagenet_subsets

[XY-ATOE]

Yes, I used the same classed list.

[Hzzone]

OK, I will check it later when I am available.

[XY-ATOE]

Thanks a lot