norlab-ulaval / mask_bev

Source code for "MaskBEV: Joint Object Detection and Footprint Completion for Bird's-eye View 3D Point Clouds"
MIT License
16 stars 3 forks source link

About "Testing file" and Visualization #3

Open HaiCLi opened 1 month ago

HaiCLi commented 1 month ago

Hey,

Is there any commands of testing and visualization?

Thx!

HaiCLi commented 1 month ago

after "PYTHONPATH=. python3 mask_bev_test/evaluation/test_kitti_eval.py" "PYTHONPATH=. python3 mask_bev_figures/test_figures.py"

There is no any outputs.

willGuimont commented 1 month ago

Hi,

I've updated the scripts. The new procedure to run the tests is in docs/TESTING.md

PYTHONPATH=. python3.10 mask_bev_figures/test_figures.py
HaiCLi commented 3 weeks ago

Hi,

I've updated the scripts. The new procedure to run the tests is in docs/TESTING.md

PYTHONPATH=. python3.10 mask_bev_figures/test_figures.py

Hi,

Thanks for your reply! I encountered two areas that confused me. Could you help me with them?

1: Is the "datamoudle" automatically split the dataset already? I found all the "train, validation, test" areas use the same datamoudle as input. Besides, the validate and test commands are both under is_testing situation.

image

image

2: The augmentation part just use 10 samples for augemntation? image

Although this work is based on Swintransformer, it still a complex project. Thanks for your contribution.

willGuimont commented 3 weeks ago

Hi,

1. For your first question, yes, the datamodule automatically split the dataset. You can see the code in that does the split in mask_bev/datasets/kitti/kitti_data_module.py and mask_bev/datasets/semantic_kitti/semantic_kitti_mask_data_module.py. PyTorch Lightning allows you to write these three functions that define the dataset splits:

def train_dataloader(self):
    return DataLoader(self._train_dataset, batch_size=self._batch_size, num_workers=self._num_workers,
                      pin_memory=self._pin_memory, shuffle=self._shuffle_train, drop_last=True,
                      collate_fn=self._collate_fn)

def val_dataloader(self):
    return DataLoader(self._valid_dataset, batch_size=self._batch_size, num_workers=self._num_workers,
                      pin_memory=self._pin_memory, shuffle=False, drop_last=True, collate_fn=self._collate_fn)

def test_dataloader(self):
    return DataLoader(self._test_dataset, batch_size=self._batch_size, num_workers=self._num_workers,
                      pin_memory=self._pin_memory, shuffle=False, drop_last=False, collate_fn=self._collate_fn)

Afterwards, the trainer.fit, trainer.validate and trainer.test function will take the corresponding DataLoader for each training phase.

For KITTI, the splits are defined using train.txt and val.txt in the dataset. For SemanticKITTI, the splits are defined in configs/semantic_kitti/semantic-kitti.yaml.

2. For your second question, the object sampler will add up to 10 cars from KITTI to a point cloud as data augmentation. In other words, it will take instances from KITTI and paste them inside the point clouds during training to augment the number of instances per scan. It won't overlap with existing instances in a scan, so it can only add cars to a particular LiDAR scan.

The code applying that augmentation can be found in mask_bev/augmentations/kitti_mask_augmentations.py. The function make_augmentation does the mapping between the names in the yaml file and the augmentation classes.

Hope it helps!

HaiCLi commented 1 week ago

Hi,

1. For your first question, yes, the datamodule automatically split the dataset. You can see the code in that does the split in mask_bev/datasets/kitti/kitti_data_module.py and mask_bev/datasets/semantic_kitti/semantic_kitti_mask_data_module.py. PyTorch Lightning allows you to write these three functions that define the dataset splits:

def train_dataloader(self):
    return DataLoader(self._train_dataset, batch_size=self._batch_size, num_workers=self._num_workers,
                      pin_memory=self._pin_memory, shuffle=self._shuffle_train, drop_last=True,
                      collate_fn=self._collate_fn)

def val_dataloader(self):
    return DataLoader(self._valid_dataset, batch_size=self._batch_size, num_workers=self._num_workers,
                      pin_memory=self._pin_memory, shuffle=False, drop_last=True, collate_fn=self._collate_fn)

def test_dataloader(self):
    return DataLoader(self._test_dataset, batch_size=self._batch_size, num_workers=self._num_workers,
                      pin_memory=self._pin_memory, shuffle=False, drop_last=False, collate_fn=self._collate_fn)

Afterwards, the trainer.fit, trainer.validate and trainer.test function will take the corresponding DataLoader for each training phase.

For KITTI, the splits are defined using train.txt and val.txt in the dataset. For SemanticKITTI, the splits are defined in configs/semantic_kitti/semantic-kitti.yaml.

2. For your second question, the object sampler will add up to 10 cars from KITTI to a point cloud as data augmentation. In other words, it will take instances from KITTI and paste them inside the point clouds during training to augment the number of instances per scan. It won't overlap with existing instances in a scan, so it can only add cars to a particular LiDAR scan.

The code applying that augmentation can be found in mask_bev/augmentations/kitti_mask_augmentations.py. The function make_augmentation does the mapping between the names in the yaml file and the augmentation classes.

Hope it helps!

Thanks for your reply!

I have three extra questions: 1: For the KITTI dataset there just exist "training" and "validation" two segments. There is no "testing" dataloader and related txt file. Even I view the script under "mask_bev_test". The testing script just uses the "training" dataloader. It confused me a lot. Screenshot from 2024-11-07 00-53-46 Screenshot from 2024-11-07 00-53-15 Screenshot from 2024-11-07 00-51-47 Screenshot from 2024-11-06 23-01-54

2: How to use the testing scripts of this project? It seems there lack of instructions about how to test the results. I found some scripts under "mask_bev_test" folder. Could you provide some guidance about it? Screenshot from 2024-11-07 00-50-19

3: How to visualize the results as you insert them into the paper. I found somethings that might be visualization scripts. But still needs further explanations.

Screenshot from 2024-11-07 00-49-03 Screenshot from 2024-11-04 21-17-57 Screenshot from 2024-11-04 21-17-04 Screenshot from 2024-11-04 21-16-27

HaiCLi commented 1 week ago

Besides, how to generate this .pkl file? After training, I got the checkpoint file.

Screenshot from 2024-11-07 18-52-25

willGuimont commented 4 days ago

1: The tests in mask_bev_test are unit tests to ensure the dimensions of the tensors are correct. I used them while developping the model to debug it. Here, I just used the training set to test the that the data loader was returning the correct format.

For KITTI, the annotations for the kitti_testing split are not available. I thus took both kitti_train and kitti_validation for my experiments. I then split kitti_train into my own train and trainval using the split defined in the dataset. kitti_validation was used as testing for my model. It should be noted that the model wasn't trained or validated on data from kitti_validation, only kitti_train.

2: The testing scripts can be run using Python's unittest library:

python3 -m unittest discover mask_bev_test

3: The visualisation were generated with the code in mask_bev_figures, we then made the figures using Inkscape. You can run the code to generate them with

# change test_semantic_kitti_2 to the code to run
python mask_bev_figures/test_figures.py TestFigures.test_semantic_kitti_2

4. For the pkl file, the code is commented in the function validation_step from mask_bev/mask_bev_module.py here: https://github.com/norlab-ulaval/mask_bev/blob/main/mask_bev/mask_bev_module.py#L341

print('Writing validation output')
log_path = pathlib.Path('~/Datasets/KITTI/output_val_01').expanduser()
log_path.mkdir(exist_ok=True)
file_path = log_path / f'{batch_idx}.pkl'
with open(file_path, 'wb') as f:
    c_masks = []
    for i in range(masks[-1][0].shape[0]):
        c = cls[-1][0][i].argmax()
            if c > 0:
                c_masks.append(masks[-1][0][i].detach().cpu().numpy())
    pickle.dump((metadata, c_masks), f)