Loss not decrease on Lizard dataset

[Lewislou]

Hi, I tried to train the cellvit-sam-h or cellvit-vit256 on the lizard dataset for nuclei classification, but consider all the tissue types are the same for convenience. But the loss can not decrease. Why is that? Are there any configurations that I need to change? I have already set all the parameters as the value mentioned in the paper.

Here is the loss:

Here is the full settings: logging: mode: online project: cellvit_brca notes: '' log_comment: null tags:

• tag1
• tag2
• '...' wandb_dir: /logs/ log_dir: /logs/ level: debug log_images: false group: None run_id: 9w4nt22v wandb_file: 9w4nt22v random_seed: 19 gpu: 0 data: dataset: lizard dataset_path: /lizard/npy/ train_folds:
• '...' val_split: null val_folds: null test_folds: null num_nuclei_classes: 7 num_tissue_classes: 1 model: backbone: SAM-H pretrained_encoder: /checkpoints/sam-vit-h.pth pretrained: null embed_dim: 1280 input_channels: 3 depth: 32 num_heads: 16 extract_layers: null shared_skip_connections: null loss: nuclei_binary_map: bce: loss_fn: xentropy_loss weight: 1 dice: loss_fn: dice_loss weight: 1 focaltverskyloss: loss_fn: FocalTverskyLoss weight: 1 training: batch_size: 8 epochs: 150 unfreeze_epoch: 0 attn_drop_rate: 0.1 drop_rate: 0.25 optimizer: AdamW optimizer_hyperparameter: lr: 0.001 betas:
  • 0.85
  • 0.9 weight_decay: 0.001 early_stopping_patience: null scheduler: scheduler_type: exponential sampling_strategy: random sampling_gamma: 1 mixed_precision: false transformations: randomrotate90: p: 0.5 horizontalflip: p: 0.5 verticalflip: p: 0.5 downscale: p: 0.15 scale: 0.5 blur: p: 0.2 blur_limit: 10 gaussnoise: p: 0.25 var_limit: 50 colorjitter: p: 0.2 scale_setting: 0.25 scale_color: 0.1 superpixels: p: 0.1 zoomblur: p: 0.1 randomsizedcrop: p: 0.1 elastictransform: p: 0.2 normalize: mean:
  • 0.5
  • 0.5
  • 0.5 std:
  • 0.5
  • 0.5
  • 0.5 run_sweep: false agent: null dataset_config: tissue_types: cell: 0 nuclei_types: Background: 0 Neoplastic: 1 Inflammatory: 2 Connective: 3 class4: 4 class5: 5 class6: 6

[FabianHoerst]

We have not trained on the Lizzard dataset, thus I cannot give you any advice on how to train on Lizzard. I assume you need to tweak the hyperparameters to find an appropriate setting. For Lizzard, the tissue type prediction is also not necessary.

Other than that, you should follow common Machine Learning project steps to find the best setting and eliminate common errors:

• Tune hyperparameters
• Test with augmentations
• Make sure the data is provided in an appropriate way
• Check if metrics needs to be adapted for the new dataset
• Check if network is training and no weights are frozen
• Check if labels are in the correct format

[FabianHoerst]

:bangbang: We are currently fixing a bug in the training pipeline that prevents the network from training. Maybe this could cause to the problem when training lizzard :bangbang:

If you find a bug, please report it to us as fast as possible

[FabianHoerst]

The bug has been fixed. Please try again. Otherwise, reopen the issue.