search

WeijingShi / Point-GNN

Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud, CVPR 2020.
MIT License
539 stars 112 forks source link

question about train #66

Open typhoonlee opened 3 years ago

typhoonlee commented 3 years ago

Hi，Thank you for your patient answer last time~ What type of GPU did you use during the training process? How many GPUs did you use? The efficiency is very slow during my training process.

WeijingShi commented 3 years ago

Hi @typhoonlee, the default train config uses two 11GB GPUs. A pair of GTX1080ti or a pair of RTX2080ti should be fine. And yes, the training takes around a week. If you have more GPUs, you can set the NUM_GPU in the train_config to a lager number. Note, the default batch size is 4. If you go beyond 4 GPUs, you will need to increase the batch size.

typhoonlee commented 3 years ago

Hello, I'm sorry to disturb you. What is the difference between run.py and eval.py and the offline package kitti_native_evaluation, and what is the relationship between them? What are their respective results? What should I do if I want to get experimental results for comparison with the paper?

WeijingShi commented 3 years ago

Hi @typhoonlee, the print out from run.py and eval.py are more or less internal/debug metrics related to this work. To compare with other works, kitti_native_evaluation is the right way to go.

Print out from run.py and eval.py: Accuracy and Recall are at point level, and car class is splited to front view car and side view car subclasses.

kitti_native_evaluation: Kitti 3D and BEV object detection scores (mAP), used in the paper.

Hope this helps. Thank you.

typhoonlee commented 3 years ago

Thank u very much！I got it and i have another question：How to get this figure? image

WeijingShi commented 3 years ago

Hi @typhoonlee, those figures in paper are drawn using OpenCV (boxes in the rgb) and Open3D (point cloud with colors). Check the visualization case VISUALIZATION_LEVEL == 2 in run.py for some examples. Thanks,

typhoonlee commented 3 years ago

Thank you very much, I got these pictures according to the method you said. But why is it different from the paper？ image

typhoonlee commented 3 years ago

image And such an error statement will appear when the program is running, how can I solve this?

WeijingShi commented 3 years ago

Hi @typhoonlee, For the figures in the paper, we modified the visualization a bit to make it clean.

To remove the "lines", you just need to delete the graph_line_set in custom_draw_geometry_load_option() call.

To remove the color of the points or add custom color for each point, simply modify pcd.colors to your desired rgb color, e.g. setting all point to black pcd.colors = open3d.Vector3dVector(np.zeros((NUM_POINTS, 3))

Not quite sure about the warning, it seems related to pre-set the viewing angle though. If it does not break the code, it might be fine.

Sorry for the delayed reply. Thanks,

typhoonlee commented 3 years ago

Thank you very much~ and i have another question: In the training , the vehicle and pedestrian categories are trained separately, and different pre-training weight files are obtained. Then in the inference phase, if I use checkpoints/car_auto_T3_train/, can I still detect traveling humans at the same time?

abhigoku10 commented 3 years ago

@WeijingShi what does the multiple line from the bounding box signify!!

WeijingShi commented 3 years ago

Thank you very much~ and i have another question: In the training , the vehicle and pedestrian categories are trained separately, and different pre-training weight files are obtained. Then in the inference phase, if I use checkpoints/car_auto_T3_train/, can I still detect traveling humans at the same time?

Hi @typhoonlee, unfortunately, the pre-trained model detects car and ped separately. Checkpoints/car_auto_T3_train only detects cars.

WeijingShi commented 3 years ago

@WeijingShi what does the multiple line from the bounding box signify!!

Hi @abhigoku10, the lines are the graph connections to the node which is outputting the box.

typhoonlee commented 3 years ago

Thank you very much~ and i have another question: In the training , the vehicle and pedestrian categories are trained separately, and different pre-training weight files are obtained. Then in the inference phase, if I use checkpoints/car_auto_T3_train/, can I still detect traveling humans at the same time?

Hi @typhoonlee, unfortunately, the pre-trained model detects car and ped separately. Checkpoints/car_auto_T3_train only detects cars.

Thank you for your patience to answer.！Why is there a green box in the final prediction graph in the paper for cars, red for pedestrians, and blue for cyclists? How should I get the prediction results of all categories in a graph? Does it need to be retrained?

And...Why are there only car results and no pedestrians and cyclists in the results obtained by using kitti_native_evaluation, but there are still two types of 3DmAP results in the paper. How do you get them?

typhoonlee commented 3 years ago

Can you share the code for the result visualization? It will be very helpful to my study, thank you very much!

WeijingShi commented 3 years ago

Hi @typhoonlee, we provide ped_cyl_auto_T3_trainval checkpoint. You can test it in Kitti test split. Because the test split has no label. You need to submit the results to Kitti website to get the mAP number. If you need the accuracy on val split, you need to retrain it using train split only.

The qualitative visualization is the combined results of a car detection model and a ped-cyl model. The visualization code is drawn based on Open3D. You can select the points within the detection box and set their color in pcd.colors as desired.

The following function may be helpful: https://github.com/WeijingShi/Point-GNN/blob/48f3d79d5b101d3a4b8439ba74c92fcad4f7cab0/dataset/kitti_dataset.py#L143

typhoonlee commented 3 years ago

Hi @typhoonlee, we provide ped_cyl_auto_T3_trainval checkpoint. You can test it in Kitti test split. Because the test split has no label. You need to submit the results to Kitti website to get the mAP number. If you need the accuracy on val split, you need to retrain it using train split only.

The qualitative visualization is the combined results of a car detection model and a ped-cyl model. The visualization code is drawn based on Open3D. You can select the points within the detection box and set their color in pcd.colors as desired.

The following function may be helpful: https://github.com/WeijingShi/Point-GNN/blob/48f3d79d5b101d3a4b8439ba74c92fcad4f7cab0/dataset/kitti_dataset.py#L143

How do you get the visualization results after the two models are combined? Oh,i see！ Is the corresponding .txt in the /eval/data file obtained by the two pre-training models combined, and then displayed as a whole? If this is the case, how should run.py be modified? Looking forward to your reply！

WeijingShi commented 3 years ago

Hi @typhoonlee, If offline visualization is fine to you, you can just combine the results files, read them as label file for visualization: https://github.com/WeijingShi/Point-GNN/blob/48f3d79d5b101d3a4b8439ba74c92fcad4f7cab0/dataset/kitti_dataset.py#L703 https://github.com/WeijingShi/Point-GNN/blob/48f3d79d5b101d3a4b8439ba74c92fcad4f7cab0/dataset/kitti_dataset.py#L1286

If you want an online combination, it's not supported by the current code. You need to rewrite run.py to load to models together and run them at the same time.