Open madan96 opened 3 years ago
ZwwWayne
commented
3 years ago Hi @madan96 , Thanks for your kind suggestion. We will simplify the process and reduce the requirements of unnecessary data structure. For now, you might need to re-implement a dataset class that does not need these things.
EricWiener
commented
3 years ago @madan96 is there any chance you could please share how you prepared your dataset to use with MMDetection3D. I'm in the same situation you are in
divyanshj16
commented
3 years ago @madan96 I am working on something similar. I have drone lidar data and 3d bboxes. My data is in the form of las files and I have annotated 3d bounding boxes for cars. I know which boxes belong to which las files. @madan96 Were you able to do training? Converting to KITTI format is not very straight forward.
In addition to that, the point clouds here have a dimension of Nx4, I am not sure how is it 4, I only have XYZ. I saw here that: https://github.com/open-mmlab/mmdetection3d/blob/63bdad9808b9e99fa9700b271d0b51376bd3fcac/mmdet3d/datasets/pipelines/loading.py#L328 it can take be arbitrary number of dimensions, how is that possible? @ZwwWayne
madan96
commented
3 years ago Hi @divyanshj16, I never tried training the network and went with another approach I was trying out. I have this branch on my fork where I committed the changes I made according to @ZwwWayne's suggestion. I remember I was able to run this code with my dataset for an epoch or 2. My suggestion would be to try converting the las files to bin or write a different LoadPointsFromFile function for your pointcloud files.
The Nx4 is for XYZI, where I stands for intensity. I think you'd be better off writing a different loading method and updating its name in the configs, along with the dimension of your cloud.
Unfortunately, I don't think I have any more information about this since it's been a while since I used this package. Good luck!
clytze0216
commented
3 years ago t Hi,@madan96.sorry for my bother. I also want to costom my own datasets which only contain anno and pointclouds. I want to ask you if I could only replace the velodyne and label_2 to train the model? Looking forward to your reply. Thansk again.
curiosity654
commented
3 years ago @madan96 I am working on something similar. I have drone lidar data and 3d bboxes. My data is in the form of las files and I have annotated 3d bounding boxes for cars. I know which boxes belong to which las files. @madan96 Were you able to do training? Converting to KITTI format is not very straight forward.
In addition to that, the point clouds here have a dimension of Nx4, I am not sure how is it 4, I only have XYZ. I saw here that:
it can take be arbitrary number of dimensions, how is that possible? @ZwwWayne
@divyanshj16
You can refer to the code for LoadPointsFromFile, where load_dim is for reshape the loaded raw data, and use_dim is for indexing the dimensions you need.
divyanshj16
commented
3 years ago Hi @madan96 Thanks for your help. I have been able to create custom dataset from my las files. I wanted to know how are you setting this point cloud range and voxel size? https://github.com/madan96/mmdetection3d/blob/c793c61c17562974ba8134ee086b95199ea33f06/configs/_base_/datasets/base-data.py#L5
divyanshj16
commented
3 years ago @ZwwWayne @madan96 Please help with my issue below.
I am able to train the model and the loss is also converging very nicely. I trained on custom datasets and annotations. I am training in min-max setting without yaw.
I did this by implementing a custom dataset and use the model build from build_model function. Below is the loss table and loss curve.
However, after such good training, the results on the validation set are not good. On training I am getting reasonable results, but on validation I am getting poor results.
ground truth
prediction
ground truth
prediction
Although, the yaw was 0 in my case, still it's predicting it, mostly ~2pi, sometimes ~pi/2
My questions:
What can I do to improve performance on the validation/test set and why is it happening? Could it be because of the fact that the training loss came down so low and validation is still stuck around 2.58? Some more facts: I have relatively small data, 30 point cloud patches containing approximately each containing 1.3 million points on an average with a total of ~400 labelled cars. My validation data contains ~30 labelled cars and 4 point cloud patches with 0.5 million points on an average.
Why is it predicting Yaw?
Fungungun
commented
3 years ago @ZwwWayne @madan96 Please help with my issue below.
I am able to train the model and the loss is also converging very nicely. I trained on custom datasets and annotations. I am training in min-max setting without yaw. I did this by implementing a custom dataset and use the model build from
build_modelfunction. Below is the loss table and loss curve.epoch train_loss valid_loss time 0 4091.541260 4339.893066 01:12 1 3511.875244 3873.412598 01:14 2 3023.028320 3248.494141 01:15 3 2599.338623 2618.090332 01:16 4 2216.144775 2036.166260 01:15 5 1868.235840 1544.580811 01:16 6 1554.849365 1141.499268 01:16 7 1278.622559 819.518127 01:16 8 1042.989502 566.451904 01:16 9 846.021851 397.228149 01:16 10 682.200989 297.114288 01:16 11 546.985229 231.899872 01:16 12 436.301361 174.552322 01:16 13 346.867157 129.714447 01:16 14 275.539581 100.248444 01:16 15 219.054153 88.966576 01:16 16 174.229584 70.900986 01:16 17 138.545380 59.210781 01:16 18 110.305984 44.358696 01:16 19 88.070435 36.418659 01:16 20 70.549622 28.117123 01:16 21 56.747650 22.572691 01:16 22 45.944958 19.011368 01:16 23 37.335182 15.910639 01:16 24 30.486134 13.948141 01:16 25 25.047031 11.473155 01:16 26 20.718050 9.987444 01:16 27 17.267614 8.705861 01:16 28 14.527399 7.983851 01:16 29 12.329118 6.875288 01:16 30 10.539511 7.063208 01:16 31 9.089251 5.811657 01:16 32 7.899366 5.649652 01:16 33 6.933311 5.285494 01:16 34 6.134204 4.850323 01:16 35 5.471034 4.785600 01:16 36 4.917679 4.579283 01:16 37 4.439167 4.457599 01:16 38 4.053056 4.256866 01:16 39 3.714704 3.735523 01:16 40 3.434179 3.786665 01:16 41 3.190169 3.675888 01:16 42 2.981537 3.819055 01:16 43 2.785687 3.371310 01:16 44 2.618291 3.546071 01:16 45 2.475881 3.480840 01:16 46 2.339902 2.912966 01:16 47 2.233608 3.440542 01:16 48 2.134134 3.117487 01:16 49 2.050812 3.134299 01:16 50 1.956867 3.031329 01:16 51 1.870409 2.786792 01:16 52 1.788638 2.800385 01:16 53 1.736393 3.059615 01:16 54 1.686984 2.619216 01:16 55 1.642131 2.836412 01:16 56 1.597658 2.705651 01:16 57 1.553393 2.757333 01:16 58 1.518408 2.360894 01:16 59 1.477399 4.827517 01:16 60 1.455092 3.272039 01:16 61 1.423524 2.625373 01:16 62 1.382070 2.436401 01:16 63 1.343764 2.449071 01:16 64 1.298866 2.415044 01:17 65 1.247886 2.494784 01:17 66 1.214924 2.600033 01:17 67 1.187451 2.414974 01:16 68 1.156835 2.469126 01:17 69 1.122727 2.689886 01:16 70 1.079685 2.463716 01:17 71 1.036399 2.303868 01:17 72 0.989412 2.355410 01:17 73 0.947672 2.473300 01:17 74 0.904058 2.421267 01:16 75 0.865463 2.577287 01:17 76 0.827695 2.406321 01:17 77 0.794519 2.514960 01:17 78 0.767848 2.357939 01:17 79 0.743575 2.441958 01:17 80 0.708842 2.554681 01:17 81 0.675347 2.466911 01:17 82 0.642671 2.425339 01:17 83 0.612111 2.494488 01:16 84 0.574456 2.569280 01:17 85 0.545474 2.516769 01:17 86 0.518124 2.535384 01:17 87 0.492167 2.539050 01:17 88 0.463795 2.564949 01:17 89 0.435068 2.553125 01:17 90 0.409151 2.582814 01:17 91 0.386457 2.587133 01:17 92 0.367280 2.591264 01:17 93 0.351287 2.593058 01:17 94 0.338143 2.591820 01:17 95 0.327476 2.591178 01:17 96 0.318956 2.590443 01:17 97 0.312278 2.588674 01:17 98 0.307149 2.587346 01:17 99 0.303295 2.586841 01:17
However, after such good training, the results on the validation set are not good. On training I am getting reasonable results, but on validation I am getting poor results.
training set example
validation set example
Although, the yaw was 0 in my case, still it's predicting it, mostly ~2pi, sometimes ~pi/2
My questions:
- What can I do to improve performance on the validation/test set and why is it happening? Could it be because of the fact that the training loss came down so low and validation is still stuck around 2.58? Some more facts: I have relatively small data, 30 point cloud patches containing approximately each containing 1.3 million points on an average with a total of ~400 labelled cars. My validation data contains ~30 labelled cars and 4 point cloud patches with 0.5 million points on an average.
- Why is it predicting Yaw?
hi did you train on point cloud data only? did you convert the coordinate system to the kitti coordinate system? I really appreciate if you could share your code on training custom dataset.
Thanks
My dataset contains pointcloud files and 3D bbox annotations in LiDAR coordinates. I looked into converting my data into the KITTI format but I see that preparing KITTI dataset for training requires the rotation and transformation matrices from
calibfiles and a separate images folder (which is probably optional to have). Is there a way to modify the KITTI dataset class and related classes/functions to get this to work w/o the calibration files and images?Or perhaps, any other dataset classes that might be easier to modify to use my dataset with?