jlianglab / BenchmarkTransformers

Other
21 stars 10 forks source link

Error while testing pretrained model on ChestXray14 dataset with RuntimeWarnings (Mean of empty slice) #8

Closed sushanthseela closed 1 month ago

sushanthseela commented 1 month ago

I encountered an issue while testing the pretrained model on the ChestXray14 dataset using the provided benchmark program. Here are the details:

Environment:

Command used to test: python main_classification.py --data_set ChestXray14 \ --model swin_base \ --init simmim \ --pretrained_weights /mnt/flash/BenchmarkTransformers/simmim_swinb_ImageNet_Xray926k.pth \ --data_dir /mnt/flash/BenchmarkTransformers/dataset \ --train_list /mnt/flash/BenchmarkTransformers/dataset/Xray14_train_official.txt \ --val_list /mnt/flash/BenchmarkTransformers/dataset/Xray14_val_official.txt \ --test_list /mnt/flash/BenchmarkTransformers/dataset/Xray14_test_official.txt \ --lr 0.01 --opt sgd --epochs 200 --warmup-epochs 0 --batch_size 64 --mode test

Error Message: I received the following error message when running the script:

{'GPU': None, 'model_name': 'swin_base', 'init': 'simmim', 'pretrained_weights': '/mnt/flash/BenchmarkTransformers/simmim_swinb_ImageNet_Xray926k.pth', 'num_class': 14, 'data_set': 'ChestXray14', 'normalization': 'imagenet', 'img_size': 224, 'img_depth': 3, 'data_dir': '/mnt/flash/BenchmarkTransformers/dataset', 'train_list': '/mnt/flash/BenchmarkTransformers/dataset/Xray14_train_official.txt', 'val_list': '/mnt/flash/BenchmarkTransformers/dataset/Xray14_val_official.txt', 'test_list': '/mnt/flash/BenchmarkTransformers/dataset/Xray14_test_official.txt', 'mode': 'test', 'batch_size': 64, 'epochs': 200, 'exp_name': '', 'opt': 'sgd', 'opt_eps': 1e-08, 'opt_betas': None, 'clip_grad': None, 'momentum': 0.9, 'weight_decay': 0.0, 'sched': 'cosine', 'lr': 0.01, 'lr_noise': None, 'lr_noise_pct': 0.67, 'lr_noise_std': 1.0, 'warmup_lr': 1e-06, 'min_lr': 1e-05, 'decay_epochs': 30, 'warmup_epochs': 0, 'cooldown_epochs': 10, 'decay_rate': 0.5, 'patience': 10, 'early_stop': True, 'num_trial': 1, 'start_index': 0, 'clean': False, 'resume': False, 'workers': 8, 'print_freq': 50, 'test_augment': True, 'anno_percent': 100, 'device': 'cuda', 'activate': 'Sigmoid', 'uncertain_label': 'LSR-Ones', 'unknown_label': 0} start testing.....

Disease = ['Atelectasis', 'Cardiomegaly', 'Effusion', 'Infiltration', 'Mass', 'Nodule', 'Pneumonia', 'Pneumothorax', 'Consolidation', 'Edema', 'Emphysema', 'Fibrosis', 'Pleural_Thickening', 'Hernia'] All trials: mAUC = [] /mnt/flash/.pyenv/versions/3.10.5/lib/python3.10/site-packages/numpy/core/fromnumeric.py:3504: RuntimeWarning: Mean of empty slice. return _methods._mean(a, axis=axis, dtype=dtype, /mnt/flash/.pyenv/versions/3.10.5/lib/python3.10/site-packages/numpy/core/_methods.py:129: RuntimeWarning: invalid value encountered in scalar divide ret = ret.dtype.type(ret / rcount) Mean AUC over All trials: = nan /mnt/flash/.pyenv/versions/3.10.5/lib/python3.10/site-packages/numpy/core/_methods.py:206: RuntimeWarning: Degrees of freedom <= 0 for slice ret = _var(a, axis=axis, dtype=dtype, out=out, ddof=ddof, /mnt/flash/.pyenv/versions/3.10.5/lib/python3.10/site-packages/numpy/core/_methods.py:163: RuntimeWarning: invalid value encountered in divide arrmean = um.true_divide(arrmean, div, out=arrmean, /mnt/flash/.pyenv/versions/3.10.5/lib/python3.10/site-packages/numpy/core/_methods.py:198: RuntimeWarning: invalid value encountered in scalar divide ret = ret.dtype.type(ret / rcount) STD over All trials: = nan

Screenshot: Here is a screenshot of the error message:

Error_RuntimeWarnings

Question: Could you please confirm the correct Python version and any additional dependencies or configurations required to run this benchmark program?

Mda233 commented 1 month ago

I see you are using this checkpoint "simmim_swinb_ImageNet_Xray926k.pth" for testing (--mode test). This is a self-supervised pretrained model that needs to be fine-tuned. Have you fine-tuned the model in the training mode? If not, you can delete the "--mode test" at the end of your command.

As for the python version, I have experimented with Python 3.6 to 3.8 versions. And the below configuration is what I recently tried:

pytorch==1.8.0 torchvision==0.9.0 torchaudio==0.8.0 cudatoolkit=11.1 -c pytorch -c conda-forge numpy==1.21.2 tqdm scikit-image==0.16.2 scikit-learn==0.22.2.post1 simpleitk==2.0.2 scipy pydicom yacs einops opencv-python==4.2.0.32 timm==0.5.4 transformers==4.1.1 albumentations==0.4.3 imgaug Pillow pretrainedmodels