Using custom data using IMNET option

[NouranFadlallah]

Hello, I'm trying to train the model on a custom medical dataset with two classes (normal and abnormal) and I'm trying to use IMNET option since it loads data from specified directory. However, the model doesn't seem to train (Accuracy is always around 55%). While debugging I noticed the loaded images are totally black.

when running this code on colab:

!python -m torch.distributed.launch --use_env /content/convit/main.py --epochs 50 --mixup 0.8 --model convit_base --drop 0.7 --batch-size 32 --nb_classes 2 --output_dir /content/drive/MyDrive/models/convit/ --data-path /content/drive/MyDrive/ct_data/RawConvit/

I added this code (in main.py after defining data_loader_train) to check the generated images:

    import matplotlib.pyplot as plt
    train_features, train_labels = next(iter(data_loader_train))
    img = train_features[0].squeeze().permute(1, 2, 0)
    label = train_labels[0]
    plt.imshow(img, cmap="gray")
    plt.savefig('img.png')
    print(np.amax(img.numpy()))
    print(np.amin(img.numpy()))

where img.png is a totally black image and the print commands give -1.1072767 and -2.117904

My dataset has the following structure, and the code reads the number of classes / images correctly:

/RawConvit/  
  train/  
    abnormal/  
      img_1.jpg 
    normal/
      img_2.jpg
  val/
    abnormal/
      img_3.jpg
    normal/
      img_4.jpg

[sdascoli]

Hi, this might come from the way your images are normalized. By default the code assumes images are in ImageNet format and normalizes them to ImageNet standard deviation (see https://github.com/facebookresearch/convit/blob/main/datasets.py#L193). Perhaps adjusting the way the data is processed in datasets.py could help ?

[NouranFadlallah]

It did! I also needed to adjust the autoaugment, color jitter and cutmix parameters to make the pre-processing suitable for medical images. Now it's finally training. Thanks!

[GoelBiju]

Hi @NouranFadlallah and @sdascoli, I am trying to test out ConViT with my own custom dataset (based on cotton leaves). It is composed of two classes (healthy and diseased).

I ran into a couple of issues getting ConViT to work in Colab. I tried what @NouranFadlallah did and set up my environment, but initialising the ImageNetDataset with the custom data path seems to failing since it is returning that the object does not have a _find_classes attribute.

I ran the following command:

!python -m torch.distributed.launch --use_env /content/convit/main.py --epochs 50 --mixup 0.8 --model convit_base --drop 0.7 --batch-size 32 --nb_classes 2 --output_dir /content/drive/MyDrive/PROJ518/models/convit/ --data-path "/content/drive/MyDrive/PROJ518/Cotton Dataset/data/"

The error was the following:

/usr/local/lib/python3.7/dist-packages/torch/distributed/launch.py:186: FutureWarning: The module torch.distributed.launch is deprecated
and will be removed in future. Use torchrun.
Note that --use_env is set by default in torchrun.
If your script expects `--local_rank` argument to be set, please
change it to read from `os.environ['LOCAL_RANK']` instead. See 
https://pytorch.org/docs/stable/distributed.html#launch-utility for 
further instructions

  FutureWarning,
| distributed init (rank 0): env://
Namespace(aa='rand-m9-mstd0.5-inc1', batch_size=32, clip_grad=None, color_jitter=0.4, cooldown_epochs=10, cutmix=1.0, cutmix_minmax=None, data_path='/content/drive/MyDrive/PROJ518/Cotton Dataset/data/', data_set='IMNET', decay_epochs=30, decay_rate=0.1, device='cuda', dist_backend='nccl', dist_url='env://', distributed=True, drop=0.7, drop_block=None, drop_path=0.1, embed_dim=48, epochs=50, eval=False, gpu=0, inat_category='name', input_size=224, local_up_to_layer=10, locality_strength=1.0, lr=0.0005, lr_noise=None, lr_noise_pct=0.67, lr_noise_std=1.0, min_lr=1e-05, mixup=0.8, mixup_mode='batch', mixup_prob=1.0, mixup_switch_prob=0.5, model='convit_base', model_ema=False, model_ema_decay=0.99996, model_ema_force_cpu=False, momentum=0.9, nb_classes=2, num_workers=10, opt='adamw', opt_betas=None, opt_eps=1e-08, output_dir='/content/drive/MyDrive/models/convit/', patience_epochs=10, pin_mem=True, pretrained=False, rank=0, recount=1, remode='pixel', repeated_aug=True, reprob=0.25, resplit=False, resume='', sampling_ratio=1.0, save_every=None, sched='cosine', seed=0, smoothing=0.1, start_epoch=0, train_interpolation='bicubic', warmup_epochs=5, warmup_lr=1e-06, weight_decay=0.05, world_size=1)
Path:  /content/drive/MyDrive/PROJ518/Cotton Dataset/data/train
Number of classes:  2
Traceback (most recent call last):
  File "/content/convit/main.py", line 395, in <module>
    main(args)
  File "/content/convit/main.py", line 194, in main
    dataset_train, args.nb_classes = build_dataset(is_train=True, args=args)
  File "/content/convit/datasets.py", line 151, in build_dataset
    sampling_ratio= (args.sampling_ratio if is_train else 1.), nb_classes=args.nb_classes)
  File "/content/convit/datasets.py", line 130, in __init__
    is_valid_file=is_valid_file, **kwargs)
  File "/content/convit/datasets.py", line 107, in __init__
    classes, class_to_idx = self._find_classes(self.root)
AttributeError: 'ImageNetDataset' object has no attribute '_find_classes'

The only change I made was upgrading timm to the latest version since otherwise the command would not work due to it giving an error stating: cannot import name 'container_abcs' from 'torch._six'.

I am not sure if I am missing anything obvious since I have the folders setup correctly with the data on Google Drive?