Runtime Error while training GroundingDino on custom dataset

I've managed to train the GroundingDino model on the example cat dataset that was provided in the README.

However, upon trying to modify the code on the example grounding_dino_swin-t_finetune_8xb2_20e_cat.py config file to train on a custom dataset, I obtain 'RuntimeError: each data_itement in list of batch should be of equal size'.

Given that the cat_dataset works, I believe the error would probably be in relation to the dataset, yet the only modification i've made to the dataset is in relation to the 'categories'.

Reproduction

What command or script did you run?

python mmdetection/tools/train.py configs/grounding_dino_swint-finetune.py

Did you make any modifications on the code or config? Did you understand what you have modified?

I copied the cat config file exactly but modified the following parts: (I've also checked the paths to the config files, annotation json files as well as image datasets)

_base_ = '../mmdetection/configs/mm_grounding_dino/grounding_dino_swin-t_pretrain_obj365.py'

data_root = 'shrimp_dataset/'
class_name = ('Shrimp',)
num_classes = len(class_name)
metainfo = dict(classes=class_name, palette=[(220, 20, 60)])

model = dict(bbox_head=dict(num_classes=num_classes))

train_dataloader = dict(
    dataset=dict(
        _delete_=True,
        type='CocoDataset',
        data_root=data_root,
        metainfo=metainfo,
        return_classes=True,
        filter_cfg=dict(filter_empty_gt=False),
        ann_file='annotations/coco_annotations_train.json',
        data_prefix=dict(img='train/')))

val_dataloader = dict(
    dataset=dict(
        metainfo=metainfo,
        data_root=data_root,
        ann_file='annotations/coco_annotations_val.json',
        data_prefix=dict(img='val/')))

test_dataloader = val_dataloader

val_evaluator = dict(ann_file=data_root + 'annotations/coco_annotations_val.json')
test_evaluator = val_evaluator

What dataset did you use?

ShrimpDensity Computer Vision Project This dataset was downloaded in the coco_mmdetection format. I didn't make any modifications to the annotations JSON files except the "categories": [{ "id": 1, "name": "Shrimp" }] portion modified any category_id == 2 to be 1 for the sake of testing.

Environment

sys.platform: win32 Python: 3.9.18 (main, Sep 11 2023, 14:09:26) [MSC v.1916 64 bit (AMD64)] CUDA available: True numpy_random_seed: 2147483648 GPU 0: NVIDIA GeForce RTX 3070 Ti CUDA_HOME: C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v11.8 NVCC: Cuda compilation tools, release 11.8, V11.8.89 MSVC: Microsoft (R) C/C++ Optimizing Compiler Version 19.38.33130 for x64 GCC: n/a PyTorch: 2.1.1 PyTorch compiling details: PyTorch built with:

C++ Version: 199711
MSVC 192930151
Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications
Intel(R) MKL-DNN v3.1.1 (Git Hash 64f6bcbcbab628e96f33a62c3e975f8535a7bde4)
OpenMP 2019
LAPACK is enabled (usually provided by MKL)
CPU capability usage: AVX2
CUDA Runtime 11.8
NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_90,code=sm_90;-gencode;arch=compute_37,code=compute_37
CuDNN 8.7
Magma 2.5.4
Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.8, CUDNN_VERSION=8.7.0, CXX_COMPILER=C:/cb/pytorch_1000000000000/work/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /bigobj /FS -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE /utf-8 /wd4624 /wd4068 /wd4067 /wd4267 /wd4661 /wd4717 /wd4244 /wd4804 /wd4273, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_DISABLE_GPU_ASSERTS=OFF, TORCH_VERSION=2.1.1, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF,

TorchVision: 0.16.1 OpenCV: 4.8.1 MMEngine: 0.10.1 MMDetection: 3.3.0+86bfcfa

Error traceback

01/20 14:42:28 - mmengine - INFO - Load checkpoint from https://download.openmmlab.com/mmdetection/v3.0/mm_grounding_dino/grounding_dino_swin-t_pretrain_obj365_goldg_grit9m_v3det/grounding_dino_swin-t_pretrain_obj365_goldg_grit9m_v3det_20231204_095047-b448804b.pth
01/20 14:42:28 - mmengine - WARNING - "FileClient" will be deprecated in future. Please use io functions in https://mmengine.readthedocs.io/en/latest/api/fileio.html#file-io
01/20 14:42:28 - mmengine - WARNING - "HardDiskBackend" is the alias of "LocalBackend" and the former will be deprecated in future.
01/20 14:42:28 - mmengine - INFO - Checkpoints will be saved to c:\Users\inferna\work_dirs\grounding_dino_swint-finetune.
Traceback (most recent call last):
  File "c:\Users\inferna\mmdetection\tools\train.py", line 121, in <module>
    main()
  File "c:\Users\inferna\mmdetection\tools\train.py", line 117, in main
    runner.train()
  File "c:\Users\inferna\anaconda3\envs\cv\lib\site-packages\mmengine\runner\runner.py", line 1777, in train
    model = self.train_loop.run()  # type: ignore
  File "c:\Users\inferna\anaconda3\envs\cv\lib\site-packages\mmengine\runner\loops.py", line 96, in run
    self.run_epoch()
  File "c:\Users\inferna\anaconda3\envs\cv\lib\site-packages\mmengine\runner\loops.py", line 111, in run_epoch
    for idx, data_batch in enumerate(self.dataloader):
  File "c:\Users\inferna\anaconda3\envs\cv\lib\site-packages\torch\utils\data\dataloader.py", line 630, in __next__
    data = self._next_data()
  File "c:\Users\inferna\anaconda3\envs\cv\lib\site-packages\torch\utils\data\dataloader.py", line 1345, in _next_data
    return self._process_data(data)
  File "c:\Users\inferna\anaconda3\envs\cv\lib\site-packages\torch\utils\data\dataloader.py", line 1371, in _process_data
    data.reraise()
  File "c:\Users\inferna\anaconda3\envs\cv\lib\site-packages\torch\_utils.py", line 694, in reraise
    raise exception
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "c:\Users\inferna\anaconda3\envs\cv\lib\site-packages\torch\utils\data\_utils\worker.py", line 308, in _worker_loop
    data = fetcher.fetch(index)
  File "c:\Users\inferna\anaconda3\envs\cv\lib\site-packages\torch\utils\data\_utils\fetch.py", line 54, in fetch
    return self.collate_fn(data)
  File "c:\Users\inferna\anaconda3\envs\cv\lib\site-packages\mmengine\dataset\utils.py", line 94, in pseudo_collate
    return data_item_type({
  File "c:\Users\inferna\anaconda3\envs\cv\lib\site-packages\mmengine\dataset\utils.py", line 95, in <dictcomp>
    key: pseudo_collate([d[key] for d in data_batch])
  File "c:\Users\inferna\anaconda3\envs\cv\lib\site-packages\mmengine\dataset\utils.py", line 78, in pseudo_collate
    raise RuntimeError(
RuntimeError: each data_itement in list of batch should be of equal size

open-mmlab / mmdetection

Runtime Error while training GroundingDino on custom dataset #11413