How to get the trained weights?

[sgzqc]

hi, thanks for your awesome work, i want to use the trained weights to get inference. So will you please public the trained weights ?

[TamasMatuszka]

Hi @sgzqc,

I uploaded the trained weights and config files. You will need to change the 'data_root' and 'ckpt_path' variables. Furthermore, you can fine-tune the model performance by changing the 'score_threshold' variable of the _bboxcoder dict.

[sgzqc]

@TamasMatuszka ，i have one question, the sensors of the paper are four cameras : front and back are both pinhole cameras, while left and right are both fisheye cameras. And when we trained the model, we are using six pictures to trained the model, the code undistort each fisheye picture to two undistort pictures. Then the question is that the two undistort pictures has some blind region , when we do the sampe preprocess how we can ensure detect all the objects in the fisheye image in camera-only model ?

[TamasMatuszka]

@sgzqc You can pass any arbitrary intrinsic matrices at this function call: https://github.com/aimotive/mm_training/blob/main/dataset/src/data_loader.py#L98 We mapped the fisheye cameras to the same intrinsic as the front camera's intrinsic but this is not mandatory. You can either modify the reference intrinsic matrix to have a larger FoV or you can change the yaw here: https://github.com/aimotive/mm_training/blob/main/dataset/src/data_loader.py#L175

It might require some experiments.

[WMCh]

Hi @TamasMatuszka , thanks for your awesome work! I notice that left and right camera were set to None and your paper did not mention the virtualization of fisheye cameras, referenced here: https://github.com/aimotive/mm_training/blob/43a4773f8693d775ed0ff621a3f01adfb8cbf5ae/dataset/src/loaders/camera_loader.py#L115 So were the left and right camera used during training models with camera?

[TamasMatuszka]

Hi @WMCh,

You are right, we did not use the left and right cameras due to memory constraints. This is the reason why their images are set to None. Line 114 is commented out where the proper behavior is coded. All cameras will be used if you uncomment Line 114 and comment out Line 115.

Hard coding side cameras to None is not the most elegant solution though.