how to infer a point cloud data

[SoulProficiency]

dear author, thanks for your help and i successfully build this project on ubuntu20.04. following README.md,this project can train and test on kitti format dataset but how to infer a point cloud data only？ Here is my situation：

1. prepare a road-traffic image
2. use depth-anything to generate the estimated depth map
3. leverage pseudo-lidar with fake parameter ,i generate a pseudo-lidar (point cloud)data.i check the data with python-pcl and its fine Here is my confusion: how can we infer this pseudo-lidar data with GLENet?how to visualize the GLENet output？

[Eaphan]

You could use the demo.py (https://github.com/Eaphan/GLENet/blob/main/tools/demo.py) for inference. Then you could visualize the prediction results with open3d binary. Try to do it yourself. Or you can ask the agents like chatgpt for help.

[SoulProficiency]

You could use the demo.py (https://github.com/Eaphan/GLENet/blob/main/tools/demo.py) for inference. Then you could visualize the prediction results with open3d binary. Try to do it yourself. Or you can ask the agents like chatgpt for help.

thanks GLENet can infer a point cloud data (.bin format),here is my code: python demo.py --cfg_file ./cfgs/kitti_models/GLENet_S.yaml --data_path ./data/ --ckpt ./weights/GLENet_S.pth but something wrong here:

2024-02-22 08:15:55,468   INFO  -----------------Quick Demo of OpenPCDet-------------------------
2024-02-22 08:15:55,468   INFO  Total number of samples:    1
2024-02-22 08:15:59,500   INFO  ==> Loading parameters from checkpoint ./weights/GLENet_S.pth to CPU
2024-02-22 08:15:59,549   INFO  ==> Checkpoint trained from version: pcdet+0.3.0+9eb8d96+py269869a
2024-02-22 08:15:59,736   INFO  ==> Done (loaded 165/165)
2024-02-22 08:16:00,024   INFO  Visualized sample index:    1
[{'pred_boxes': array([[28.252472 , 10.943333 , -0.2352296,  4.279696 ,  1.6428412,
         1.3712578,  1.5511806]], dtype=float32), 'pred_scores': tensor([0.4695], device='cuda:0'), 'pred_labels': tensor([1], device='cuda:0')}]
[Open3D WARNING] GLFW Error: X11: The DISPLAY environment variable is missing
[Open3D WARNING] Failed to initialize GLFW
Traceback (most recent call last):
  File "demo.py", line 112, in <module>
    main()
  File "demo.py", line 100, in main
    V.draw_scenes(
  File "/home/wxj/wxj/GLENet/tools/visual_utils/open3d_vis_utils.py", line 49, in draw_scenes
    vis.get_render_option().point_size = 1.0
AttributeError: 'NoneType' object has no attribute 'point_size'

i use xshell to connet the remote server，and

[{'pred_boxes': array([[28.252472 , 10.943333 , -0.2352296,  4.279696 ,  1.6428412,
         1.3712578,  1.5511806]], dtype=float32), 'pred_scores': tensor([0.4695], device='cuda:0'), 'pred_labels': tensor([1], device='cuda:0')}]

is the result of pred_dicts. Due to lack a screen to display，it shows me:

[Open3D WARNING] GLFW Error: X11: The DISPLAY environment variable is missing
[Open3D WARNING] Failed to initialize GLFW

but why it tells me： AttributeError: 'NoneType' object has no attribute 'point_size' i can’t find the answer on the internet(also ask help for chatGPT3.5)

[Eaphan]

Try to modify the code and comment line of "vis.get_render_option().point_size = 1.0" out. Besides, you should prepare a screen for visualization. If you run your code on a remote server, prepare a display environment. For me, I use MobaXterm which includes X server on the windows system.

[SoulProficiency]

Try to modify the code and comment line of "vis.get_render_option().point_size = 1.0" out. Besides, you should prepare a screen for visualization. If you run your code on a remote server, prepare a display environment. For me, I use MobaXterm which includes X server on the windows system.

i read .bin file and draw box locally with open3d (windows 10/11). due to pseudo-lidar data, the box is definitely wrong： but i just want to know, if there are some steps about rescale(just like normlize)? here is my estimated depth map: the location of car is here: thanks!good work!

[Windson9]

Hey @SoulProficiency. I am trying to do something similar to what you are trying. Were you able to have good inference result using your methods ?

[SoulProficiency]

Hey @SoulProficiency. I am trying to do something similar to what you are trying. Were you able to have good inference result using your methods ?

This pipeline is accessible(img->depth anything->pseudo-lidar->GLEnet) but the result is still incorrect. I am not working on this but if you want to have a good inference, we can try the following steps:

1. using a disparity map to get a depth map(the depth anything model may not feedback us a exact result)
2. define camera coordinates, lidar coordinates, and world coordinates, trying to convert the depth map to lidar data(focusing on Coordinate parameters, not the deeplearning model)