ztsrxh / RoadBEV

Codes for RoadBEV: road surface reconstruction in Bird's Eye View
MIT License
128 stars 12 forks source link

Abnormal accuracy #2

Closed xubin1994 closed 2 months ago

xubin1994 commented 2 months ago

I generated the ground truth using preprocess_gt.py and regenerated the left_half and right_half folders for the dataset,but got the metrics (stereo mode): test: abs_err:5.425, rmse:5.679, >0.5cm:99.95, This accuracy seems to be an order of magnitude greater than what is described in the article.

ztsrxh commented 2 months ago

Thanks for your interest. We have provided the GT labels generated in our environment. You can use it thus eliminating potential problems in GT generation. We found a mistake in dataset.py last week, which has been corrected now. You can update this script.

xubin1994 commented 2 months ago

I used the GT you provided, and the results I got were still the same as before。 I visualize some of the results(RoadBEV/filenames/train/20230408031625.800.pkl). The image below is an example of the training set. The black part is to projected grid into the image part, ensuring that the input is correct. image and the elevation result like that: image

xubin1994 commented 2 months ago

gt same as you, the above sample is inference result, using the checkpoints provided.

xubin1994 commented 2 months ago

About the dataset.py: I re-pulled the latest branch so there should be no problem. About the environment: I installed the same conda environment according to the readme. The only thing that may be different is: the half-resolution image was generated by myself using opencv, because the downloaded RSRD data set does not have left_half and right_half folders

ztsrxh commented 2 months ago

I do not know where you download the dataset. You can download it again on our website. Maybe channel order is changed, i.e., BGR rather than RGB

xubin1994 commented 2 months ago

I re-downloaded the dataset and the problem was solved. But I seem to have found a problem with dataset.py, in https://github.com/ztsrxh/RoadBEV/blob/ca5bea1ef3a5e18749f6e2c34718e6d93eebcdd9/utils/dataset.py#L241, It seems like a deep copy should be added here,otherwise voxel_cam_right and voxel_cam_left will be the same, see the log below: image

jiajigiser commented 2 months ago

May I ask if it is possible to export requirements. txt (a list of all libraries and versions)? My CUDA version is 11.8, and I am not sure how to install the correct library version.

ztsrxh commented 2 months ago

I re-downloaded the dataset and the problem was solved. But I seem to have found a problem with dataset.py, in

https://github.com/ztsrxh/RoadBEV/blob/ca5bea1ef3a5e18749f6e2c34718e6d93eebcdd9/utils/dataset.py#L241

, It seems like a deep copy should be added here,otherwise voxel_cam_right and voxel_cam_left will be the same, see the log below: image

Thanks for your careful check. We have fixed this bug and now deepcopy the _voxel_camleft. The ckpt weight for RoadBEV-stereo is also updated.

jiajigiser commented 2 months ago

May I ask how to visualize.pkl results(elevation result?) and how to view the results of 3D reconstruction?