About "Testing file" and Visualization

[HaiCLi]

Hey,

Is there any commands of testing and visualization?

Thx!

[HaiCLi]

after "PYTHONPATH=. python3 mask_bev_test/evaluation/test_kitti_eval.py" "PYTHONPATH=. python3 mask_bev_figures/test_figures.py"

There is no any outputs.

[willGuimont]

Hi,

I've updated the scripts. The new procedure to run the tests is in docs/TESTING.md

PYTHONPATH=. python3.10 mask_bev_figures/test_figures.py

[HaiCLi]

Hi,

I've updated the scripts. The new procedure to run the tests is in docs/TESTING.md

PYTHONPATH=. python3.10 mask_bev_figures/test_figures.py

Hi,

Thanks for your reply! I encountered two areas that confused me. Could you help me with them?

1: Is the "datamoudle" automatically split the dataset already? I found all the "train, validation, test" areas use the same datamoudle as input. Besides, the validate and test commands are both under is_testing situation.

2: The augmentation part just use 10 samples for augemntation?

Although this work is based on Swintransformer, it still a complex project. Thanks for your contribution.

[willGuimont]

Hi,

1. For your first question, yes, the datamodule automatically split the dataset. You can see the code in that does the split in mask_bev/datasets/kitti/kitti_data_module.py and mask_bev/datasets/semantic_kitti/semantic_kitti_mask_data_module.py. PyTorch Lightning allows you to write these three functions that define the dataset splits:

def train_dataloader(self):
    return DataLoader(self._train_dataset, batch_size=self._batch_size, num_workers=self._num_workers,
                      pin_memory=self._pin_memory, shuffle=self._shuffle_train, drop_last=True,
                      collate_fn=self._collate_fn)

def val_dataloader(self):
    return DataLoader(self._valid_dataset, batch_size=self._batch_size, num_workers=self._num_workers,
                      pin_memory=self._pin_memory, shuffle=False, drop_last=True, collate_fn=self._collate_fn)

def test_dataloader(self):
    return DataLoader(self._test_dataset, batch_size=self._batch_size, num_workers=self._num_workers,
                      pin_memory=self._pin_memory, shuffle=False, drop_last=False, collate_fn=self._collate_fn)

Afterwards, the trainer.fit, trainer.validate and trainer.test function will take the corresponding DataLoader for each training phase.

For KITTI, the splits are defined using train.txt and val.txt in the dataset. For SemanticKITTI, the splits are defined in configs/semantic_kitti/semantic-kitti.yaml.

2. For your second question, the object sampler will add up to 10 cars from KITTI to a point cloud as data augmentation. In other words, it will take instances from KITTI and paste them inside the point clouds during training to augment the number of instances per scan. It won't overlap with existing instances in a scan, so it can only add cars to a particular LiDAR scan.

The code applying that augmentation can be found in mask_bev/augmentations/kitti_mask_augmentations.py. The function make_augmentation does the mapping between the names in the yaml file and the augmentation classes.

Hope it helps!

[HaiCLi]

Hi,

1. For your first question, yes, the datamodule automatically split the dataset. You can see the code in that does the split in mask_bev/datasets/kitti/kitti_data_module.py and mask_bev/datasets/semantic_kitti/semantic_kitti_mask_data_module.py. PyTorch Lightning allows you to write these three functions that define the dataset splits:

def train_dataloader(self):
    return DataLoader(self._train_dataset, batch_size=self._batch_size, num_workers=self._num_workers,
                      pin_memory=self._pin_memory, shuffle=self._shuffle_train, drop_last=True,
                      collate_fn=self._collate_fn)

def val_dataloader(self):
    return DataLoader(self._valid_dataset, batch_size=self._batch_size, num_workers=self._num_workers,
                      pin_memory=self._pin_memory, shuffle=False, drop_last=True, collate_fn=self._collate_fn)

def test_dataloader(self):
    return DataLoader(self._test_dataset, batch_size=self._batch_size, num_workers=self._num_workers,
                      pin_memory=self._pin_memory, shuffle=False, drop_last=False, collate_fn=self._collate_fn)

Afterwards, the trainer.fit, trainer.validate and trainer.test function will take the corresponding DataLoader for each training phase.

For KITTI, the splits are defined using train.txt and val.txt in the dataset. For SemanticKITTI, the splits are defined in configs/semantic_kitti/semantic-kitti.yaml.

2. For your second question, the object sampler will add up to 10 cars from KITTI to a point cloud as data augmentation. In other words, it will take instances from KITTI and paste them inside the point clouds during training to augment the number of instances per scan. It won't overlap with existing instances in a scan, so it can only add cars to a particular LiDAR scan.

The code applying that augmentation can be found in mask_bev/augmentations/kitti_mask_augmentations.py. The function make_augmentation does the mapping between the names in the yaml file and the augmentation classes.

Hope it helps!

Thanks for your reply!

I have three extra questions: 1: For the KITTI dataset there just exist "training" and "validation" two segments. There is no "testing" dataloader and related txt file. Even I view the script under "mask_bev_test". The testing script just uses the "training" dataloader. It confused me a lot.

2: How to use the testing scripts of this project? It seems there lack of instructions about how to test the results. I found some scripts under "mask_bev_test" folder. Could you provide some guidance about it?

3: How to visualize the results as you insert them into the paper. I found somethings that might be visualization scripts. But still needs further explanations.

[HaiCLi]

Besides, how to generate this .pkl file? After training, I got the checkpoint file.

[willGuimont]

1: The tests in mask_bev_test are unit tests to ensure the dimensions of the tensors are correct. I used them while developping the model to debug it. Here, I just used the training set to test the that the data loader was returning the correct format.

For KITTI, the annotations for the kitti_testing split are not available. I thus took both kitti_train and kitti_validation for my experiments. I then split kitti_train into my own train and trainval using the split defined in the dataset. kitti_validation was used as testing for my model. It should be noted that the model wasn't trained or validated on data from kitti_validation, only kitti_train.

2: The testing scripts can be run using Python's unittest library:

python3 -m unittest discover mask_bev_test

3: The visualisation were generated with the code in mask_bev_figures, we then made the figures using Inkscape. You can run the code to generate them with

# change test_semantic_kitti_2 to the code to run
python mask_bev_figures/test_figures.py TestFigures.test_semantic_kitti_2

4. For the pkl file, the code is commented in the function validation_step from mask_bev/mask_bev_module.py here: https://github.com/norlab-ulaval/mask_bev/blob/main/mask_bev/mask_bev_module.py#L341

print('Writing validation output')
log_path = pathlib.Path('~/Datasets/KITTI/output_val_01').expanduser()
log_path.mkdir(exist_ok=True)
file_path = log_path / f'{batch_idx}.pkl'
with open(file_path, 'wb') as f:
    c_masks = []
    for i in range(masks[-1][0].shape[0]):
        c = cls[-1][0][i].argmax()
            if c > 0:
                c_masks.append(masks[-1][0][i].detach().cpu().numpy())
    pickle.dump((metadata, c_masks), f)