trivial solution

[akidway]

Hi, @wwyi1828

I'm trying to reproduce your result.

But when I use CluSiam (training for 50 epochs) to extract features, I found that all tiles representation are almost same.

When I used these representations to train a mil model, the result auc is always around 0.5.

To train a CluSiam model, I used commands as below:

 python3 main.py \
--model_type clusiam \
--num_clusters 100 \
--feat_size 2048 \
--epochs 50 \
--alpha 0.5 \
--fix_pred_lr \
--batch_size 512 \
--num_worker 32 \
--save_path /work/lzh/model/20x-clusiam-full-precision \
--log_dst /work/lzh/model/20x-clusiam/20x-clusiam-full-precision \
/work/lzh/data/tiles/memory/tiles20x/training

The training data are 2.5 millions .jpeg files (C16 processed with Clam).

Is there any trick I missed? Could you please give some advice?

[akidway]

While when I used CluSiam trained for 20 epochs, trivial solution seems disappeared.

And when these features (extracted by 20 epochs CluSiam) used to train a mil model, the result auc also seems normal. In valid set of training data, the auc could reach about 0.9.

[wwyi1828]

Hi @akidway ,

I used CLAM preprocessed datasets as well recently and observed that CluSiam wasn’t as stable as it was with the datasets used previously. I suspect this instability might be due to more redundant backgrounds affecting the pre-training, such as non-tissue regions (e.g., black bars or other artifacts at the edges of WSI) generated by CLAM pipelines.

Increasing the alpha value might address the issue you mentioned. In my In my subsequent experiences, it can further improve mil classification accuracy on the original datasets. More importantly, could you try the CluBYOL training method? I have found it to be more stable than CluSiam and highly recommend using it.

[akidway]

Thank you for your prompt response. Your advice is valuable to me. I'll give CluBYOL a try later on. Best wishes.