[Question] Defining test_labels and target_class for custom dataser

[salusanga]

Hi, I have a custom dataset for brain metastasis detection. It is a binary task, in the original dataset "1" is the FG and "0" is BG. I brought it to the nnDetection format, with the instances in the masks and the new FG class being "0".

In this context, how should I set "test_labels" and "target_class"?

Thanks a lot! Best, Sara

[mibaumgartner]

Hey,

sorry for the delay, I needed to take care of some deadlines first :)

please make sure, that you have used some heuristic to cluster the original metastasis into individual objects otherwise you will only generate a single bounding box around all of them (since the original dataset seems to be in a semantic segmentation format).

test_labels simply indicates if you are planning to split a test set which will have test labels as well. If you there is one, set it to True otherwise to False.

target_class is targeted towards patient wise evaluation with multiple classes, you can either leave it empty or set it to 0.

Best, Michael

[salusanga]

Dear Michael, thanks a lot for the explanation. Just to make sure for "test_labels": with "test set which will have test labels as well", do you mean the existence of the folder "labelsTs/" with the same classes as the training? Or something else? So far I was running my code with "test_labels=False" and it seemed to work reasonably, now I am wondering if I missed something. Best, Sara

[mibaumgartner]

Yes, if you are planning to use imagesTs & labelsTs than you should set test_labels=True, that will simply trigger the processing for the test images & labels as well (since for the evaluation of the test set the boxes need to be extracted from the test labels). nnDetection was designed for use cases where the test set labels might be unknown (e.g. during a challenge) and thus it won't automatically process the labels.

Training etc. don't use the test set - the test set is really intended as a true test set i.e. you should only predict it once at the very end of the model development. This is done by separate commends i.e. running nndet_predict and nndet_eval with the --test option.

Best, Michael

[salusanga]

• Practically, what would test_labels=True do? Would it create also a labelsTs/ folder under {my task}/preprocessed/D3V001_3d/ ? At the moment I only have there: imagesTr/, imagesTs/, and labelsTr/ . However, I already have also a labelsTs/ folder under {my task}/preprocessed/ .

• I checked the code and it seems to me that "test_labels=True" would only additionally trigger "check_data_and_label_splitted" on the test folder, just as a check.

• Should what I have not be enough, what do I need to rerun? nndet_prep?

Thanks and best. Sara

[mibaumgartner]

• Yes, it will create labelsTs in the preprocessed folder (which is basically a conversion of the format)

• Simply rerun nndet_prep which will prepare the test labels as well (https://github.com/MIC-DKFZ/nnDetection/blob/8032b8d2d4366ce0c3812ce96f747b443fa2f080/nndet/planning/experiment/utils.py#L58 automatically checks if labelsTs exist irrespective of the dataset.json)

Best, Michael

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.