search

experiencor / image-to-3d-bbox

Build a CNN network to predict 3D bounding box of car from 2D image.
https://experiencor.github.io/sdc_3d.html
237 stars 59 forks source link

The dataset format #2

Open lucasjinreal opened 6 years ago

lucasjinreal commented 6 years ago

Hi, I just found you are an expert in detection. May I raise a question about this repo?

What's the data format in predict 3d box, we using detection dataset which is bouding box or we using 3d dataset with 3d bouding box? If using label with 3d bouding box, where should we found the datasets.

experiencor commented 6 years ago

@jinfagang Input is the image patch defined by the 2D bounding box. Labels (output) are car dimension and orientation angle, estimated from 3D bounding box.

You can find the dataset with 2D bounding and 3D bounding boxes at http://www.cvlibs.net/datasets/kitti/eval_object.php?obj_benchmark=3d. I used this one in the project.

lucasjinreal commented 6 years ago

@experiencor Hi, what is the video predict data from? Seems if I want to predict on a video, I also need a calib file to calib camera, what if I want predict from a drive video record by myself?

experiencor commented 6 years ago

I got the video from here http://www.cvlibs.net/datasets/kitti/raw_data.php. Only a number of videos has labels. You need to download the right one, says "2011_09_26_drive_0084 (1.5 GB)".

You don't need the calib files if you only want to predict the orientation of the car. But you do need the calib files to overlay the predicted 3D onto the 2D images. Overlaying with calib files is also possible. The method is described in that paper. I'm not sure about this part though.

lucasjinreal commented 6 years ago

Thanks a ton! I will check that later.

jediofgever commented 6 years ago

Hi, @jinfagang did you figure out where are the labels for the video I am having the same issue just as here

radzfoto commented 6 years ago

Hi, @jinfagang , in your file 3D_Box_Regression.ipynb you have a line commented out: #model.load_weights('initial_weights.h5') Does it help to have the initial weights to shorten training? If so, can you tell me where can I download the weights?

lucasjinreal commented 5 years ago

@radzfoto I dont think that model is neccessary, just train a model from scratch. I am still training the model, after that, I will post some result about this.