Size mismatch error between images and masks AFTER loading dataset

[zacklew]

Instructions To Reproduce the Issue:

Trying to train a model using panoptic_fpn

torch.cuda.empty_cache() config_file = "COCO-PanopticSegmentation/panoptic_fpn_R_50_3x.yaml" cfg = get_cfg() cfg.MODEL.DEVICE = "cuda" if torch.cuda.is_available() else "cpu" cfg.merge_from_file(model_zoo.get_config_file(config_file)) cfg.DATASETS.TRAIN = (f"{data_name}_separated",) cfg.DATASETS.TEST = () cfg.DATALOADER.NUM_WORKERS = 0 cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(config_file)
cfg.SOLVER.IMS_PER_BATCH = 1 cfg.SOLVER.BASE_LR = 0.00025 cfg.SOLVER.MAX_ITER = 5000 cfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 16 cfg.MODEL.ROI_HEADS.NUM_CLASSES = 1 cfg.MODEL.SEM_SEG_HEAD.NUM_CLASSES = 2 cfg.MODEL.PANOPTIC_FPN.NUM_CLASSES = 1
cfg.SOLVER.AMP.ENABLED = True

MetadataCatalog.get(cfg.DATASETS.TRAIN[0]).set(thing_classes=things, stuff_classes=stuff, thing_dataset_id_to_contiguous_id={1: 0})

Example images that are causing error bad_imgs_2.zip

Expected behavior:

I've run this exact code with a different custom image set and the model ran fine. There are a few images in this image set (i.e. example images and masks below attached) that we have found to through this size error that says the masks and images are different sizes. I manually checked the dimensions of the masks and images using img properties on my PC, and it says the images and masks are the same dimensions. However, when I check the image and mask sizes after the model loads the dataset, the sizes are different. My running theory is that there is some transformation that occurs with the dataset mapper that changes the dimensions of the masks, but I'm not sure how to see if that is the case or if something else in the code is changing the dimensions of the mask and the image separately.

Any and all help trying to solve this error would be appreciated.

*Example images used in dataset in this repo: ([(https://github.com/zacklew/bad_images)])

Environment:

sys.platform linux Python 3.12.4	packaged by Anaconda, Inc.	(main, Jun 18 2024, 15:12:24) [GCC 11.2.0] numpy 1.26.4 detectron2 0.6 @/home/computational/anaconda3/lib/python3.12/site-packages/detectron2 Compiler GCC 11.2 CUDA compiler CUDA 12.5 detectron2 arch flags 8.6 DETECTRON2_ENV_MODULE PyTorch 2.3.1+cu121 @/home/computational/anaconda3/lib/python3.12/site-packages/torch PyTorch debug build False torch._C._GLIBCXX_USE_CXX11_ABI False GPU available Yes GPU 0 NVIDIA RTX A2000 12GB (arch=8.6) Driver version 555.58.02 CUDA_HOME /home/computational/anaconda3 Pillow 10.3.0 torchvision 0.18.1+cu121 @/home/computational/anaconda3/lib/python3.12/site-packages/torchvision torchvision arch flags 5.0, 6.0, 7.0, 7.5, 8.0, 8.6, 9.0 fvcore 0.1.5.post20221221 iopath 0.1.9 cv2 4.10.0

PyTorch built with:

• GCC 9.3
• C++ Version: 201703
• Intel(R) oneAPI Math Kernel Library Version 2023.1-Product Build 20230303 for Intel(R) 64 architecture applications
• Intel(R) MKL-DNN v3.3.6 (Git Hash 86e6af5974177e513fd3fee58425e1063e7f1361)
• OpenMP 201511 (a.k.a. OpenMP 4.5)
• LAPACK is enabled (usually provided by MKL)
• NNPACK is enabled
• CPU capability usage: AVX512
• CUDA Runtime 12.1
• NVCC architecture flags: -gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_90,code=sm_90
• CuDNN 8.9.2
• Magma 2.6.1
• Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=12.1, CUDNN_VERSION=8.9.2, CXX_COMPILER=/opt/rh/devtoolset-9/root/usr/bin/c++, CXX_FLAGS= -D_GLIBCXX_USE_CXX11_ABI=0 -fabi-version=11 -fvisibility-inlines-hidden -DUSE_PTHREADPOOL -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOROCTRACER -DUSE_FBGEMM -DUSE_QNNPACK -DUSE_PYTORCH_QNNPACK -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -O2 -fPIC -Wall -Wextra -Werror=return-type -Werror=non-virtual-dtor -Werror=bool-operation -Wnarrowing -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-unused-parameter -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-stringop-overflow -Wsuggest-override -Wno-psabi -Wno-error=pedantic -Wno-error=old-style-cast -Wno-missing-braces -fdiagnostics-color=always -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -fno-math-errno -fno-trapping-math -Werror=format -Wno-stringop-overflow, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=2.3.1, USE_CUDA=ON, USE_CUDNN=ON, USE_CUSPARSELT=1, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_GLOO=ON, USE_MKL=ON, USE_MKLDNN=ON, USE_MPI=OFF, USE_NCCL=1, USE_NNPACK=ON, USE_OPENMP=ON, USE_ROCM=OFF, USE_ROCM_KERNEL_ASSERT=OFF,

[github-actions[bot]]

You've chosen to report an unexpected problem or bug. Unless you already know the root cause of it, please include details about it by filling the issue template. The following information is missing: "Instructions To Reproduce the Issue and Full Logs";