Query about the dataset preparation for other dataset

[jikedream]

Hi,

First of all, thanks for your amazing work!

I try to use potsdam dataset on MulitMAE, my dataset architecture is like this：

/path/to/potsdam/ ├── train/ │ ├── rgb/ │ │ └── all/ │ │ ├── 1.jpg │ │ ├── 2.jpg │ │ └── ... │	└── semseg/ │	└── all/ │	├── 1.png │	├── 2.png │			└── depth/		└── all/	├── 1.tif │	├── 2.tif

└── val/ ├── rgb/ │ └── all/ │ ├──1.jpg │ ├── 2.jpg │ └── ... | └── semseg/ | └── all/ | ├──1.png | ├── 2.png | └── ... | └── depth/ | └── all/ | ├── 1.tif | ├── 2.tif

My yaml file looks like this：

> # NYU semseg config

# Finetune from:

finetune: 'D:/graduate study/MultiMAE-main/multimae-b_98_rgb+-depth-semseg_1600e_multivit-afff3f8c.pth' # Change me

# Input tasks

in_domains: rgb-depth
decoder_main_tasks: rgb # Can also be changed to rgb-depth
use_mask_valid: False # Requires "task" mask_valid to be saved to disk

# Architecture

model: multivit_base
patch_size: 16
num_global_tokens: 1
drop_path_encoder: 0.1
output_adapter: convnext
decoder_dim: 6144
decoder_preds_per_patch: 16
decoder_depth: 4

# Train

epochs: 1
opt: adamw
lr: 0.0001 # = 1e-4
warmup_lr: 0.000001 # = 1e-6
min_lr: 0.
warmup_epochs: 1
batch_size: 2
input_size: 512
layer_decay: 0.75

# Augmentation

aug_name: simple

# Data info

data_path: 'D:/datasets/satellite-dataset/potsdam_done/train' # Change me
eval_data_path: 'D:/datasets/satellite-dataset/potsdam_done/val' # Change me
num_classes: 150  
dataset_name: potsdam
dist_eval: True
seg_reduce_zero_label: True
eval_freq: 20

# Misc.

find_unused_params: False

# Wandb and logging

log_wandb: False # Set to True to log to Weights & Biases
wandb_project: 'multimae-finetune-semseg'
wandb_entity: null # Change if needed
wandb_run_name: 'ft_nyu_200e_multimae-b_rgb-depth'
log_images_wandb: True
log_images_freq: 20
output_dir: 'output/finetune/semseg/nyu/ft_nyu_200e_multimae-b_rgb-depth'

But when I run the run_finetuning_semseg.py it, I get this error:

RuntimeError: CUDA error: device-side assert triggered CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.

I want to know if there is something wrong with my dataset structure or if my yaml file is configured incorrectly

Best, David

[li-zp21]

Hi, I've encountered the same problem. Have you solved it?

[jikedream]

Hi, I've encountered the same problem. Have you solved it?

i set the num_classes's number to 150 then i can run. I dont know why

[li-zp21]

Hi, I've encountered the same problem. Have you solved it?

i set the num_classes's number to 150 then i can run. I dont know why

Thanks. I've also just figured that out. I think maybe you used a model pretrained on ADE20K for semseg, which has 150 num_classes instead of 133 on COCO.

[jikedream]

Hi, I've encountered the same problem. Have you solved it?

i set the num_classes's number to 150 then i can run. I dont know why

Thanks. I've also just figured that out. I think maybe you used a model pretrained on ADE20K for semseg, which has 150 num_classes instead of 133 on COCO.

absolutely right!