We provide off-the-shelf scripts for running the simulations. All you need to do is to prepare the clean datasets first and put them under a suitable folder structure.
For evaluating your 3D detector / segmentors on our corruption sets:
You can first download or generate the corruption sets. After this, you can just modify the file path in your data loader to test the model's robustness. Our corruption sets follow the same structure as the clean datasets hence no extra or particular thing is needed. All you need to do is to make some simple modifications to your evaluation code.
We will provide an easy-to-use toolkit in the future to assist the evaluation process.
Hi @qpc001, can you provide more details on which part are you trying to reproduce?
For preparing the robustness datasets:
For generating the corruption set on your own:
For evaluating your 3D detector / segmentors on our corruption sets:
Let me if you have any other questions!