search

cguindel / eval_kitti

Tools to evaluate object detection results using the KITTI dataset.
57 stars 23 forks source link

Any examples of the format of the .txt or results generator? #3

Closed ywangeq closed 6 years ago

cguindel commented 6 years ago

I cannot help you if you do not provide further details about your problem. Anyway, the expected .txt format is obviously the one used by the KITTI dataset. You can find a detailed description as a part of the object development kit which is available for download from the KITTI website.

ywangeq commented 6 years ago

Now I detect the 3D bboxs with 8 points in xyz, and the format of kitti need 16 parameters in one row, even I transfer the points to h w l x y z r and score, the other parameters still doesn't know, or I just copy them from labels?

cguindel commented 6 years ago

If I remember correctly, the only required fields are type, bbox and score. If you don't have any of the other values, you can just fill them using the default values that you can find in the matlab/writeLabels.m script from the devkit (they are basically -1, -10 or -1000, depending on the particular field).

In case you only have the 3D coordinates, you will probably need to project the points to the image by using the projection matrices to obtain the bbox values. Information about this procedure can be found in the readme.txt from the devkit. The equation is:

x = P2 R0_rect Tr_velo_to_cam * y

with the P2, R0_rect and Tr_velo_to_cam matrices available from the calibration files.

ywangeq commented 6 years ago

Thanks. Now I have got all result as the text files which followed the kitti data format, however, I still can not run the code. I create the results, list, folder in the build file, and create the second level folder 'data' under the result and put the results in the folder. But the code still cannot detect just show all zero results in the terminal and finished

cguindel commented 6 years ago

Thanks for the additional information. Let's try to identify the problem. First of all, about the zeros that you are getting in the terminal: are they under the GT STATS or DET STATS header?

cguindel commented 6 years ago

Closed due to inactivity

jkstyle2 commented 5 years ago

Hi, @ywangeq , @cguindel How can we get the field score in KITTI's format? Could you share the idea?

ywangeq commented 5 years ago

with the result，u can generate the txt follow the format of kitti ground truth. In the readme of kitti, it has told you how to format.

获取 Outlook for iOShttps://aka.ms/o0ukef

发件人: jaykay notifications@github.com 发送时间: 星期二, 四月 2, 2019 12:47 上午 收件人: cguindel/eval_kitti 抄送: Yuan WANG; Mention 主题: Re: [cguindel/eval_kitti] Any examples of the format of the .txt or results generator? (#3)

Hi, @ywangeqhttps://github.com/ywangeq , @cguindelhttps://github.com/cguindel How can we get the field score in KITTI's format? Could you share the idea?

― You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/cguindel/eval_kitti/issues/3#issuecomment-478656595, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AYluX6ObCMx18Kp9cZc7Mdaq-0XJsgKJks5vcjglgaJpZM4Rg0fF.

jkstyle2 commented 5 years ago

@ywangeq Thanks for the reply, I followed all the elements in kitti's format, except the 16th element score. Could you tell me how to get the value score?

ywangeq commented 5 years ago

When you do the object detection, you will have a regression function and classification score. Simply, all of the boxes will have a confidence score which is the 16th value.

寄件者: jaykay notifications@github.com 寄件日期: 2019年4月1日 15:18 收件者: cguindel/eval_kitti 副本: Yuan WANG; Mention 主旨: Re: [cguindel/eval_kitti] Any examples of the format of the .txt or results generator? (#3)

@ywangeqhttps://github.com/ywangeq Thanks for the reply, I followed all the elements in kitti's format, except the 16th element score. Could you tell me how to get the value score?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/cguindel/eval_kitti/issues/3#issuecomment-478828329, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AYluX9NgN5X5X20pv-c0k1Ak8oyfiG0Rks5vcsvrgaJpZM4Rg0fF.

jkstyle2 commented 5 years ago

Well, 2D Detection has a confidence score as you mentioned, but when I regress the 3D information such as H, W, L, x, y, z and yaw angle, I don't really have any confidence score of them. Is this still fine to use the 2D detection score when I try 3D evaluation?

ywangeq commented 5 years ago

Firstly, if you do the 3D detection, you should have the score, expect that you do not do the classifiy and only do the boundary box regression. If you do like this, you have to set the 16th value to be 1(I am not sure). And the 3D evaluation is the same as 2D evaluation, you just generate same kitti txt and insert corresponding xyzhwd value in the right place.

获取 Outlook for iOShttps://aka.ms/o0ukef

发件人: jaykay notifications@github.com 发送时间: 星期二, 四月 2, 2019 11:30 上午 收件人: cguindel/eval_kitti 抄送: Yuan WANG; Mention 主题: Re: [cguindel/eval_kitti] Any examples of the format of the .txt or results generator? (#3)

Well, 2D Detection has a confidence score as you mentioned, but when I regress the 3D information such as H, W, L, x, y, z and yaw angle, I don't really have any confidence score of them. Is this still fine to use the 2D detection score when I try 3D evaluation?

― You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/cguindel/eval_kitti/issues/3#issuecomment-478830503, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AYluX3DGDBkEFPJtXHZxrUNRJJENcfNjks5vcs7PgaJpZM4Rg0fF.

jkstyle2 commented 5 years ago

Thanks for the quick reply! So, what you mean is that the score should be still filled for "3D Box regression with any value", but it doesn't actually affect evaluating 3D metrics. Is that right?

Thank you very much, I'll keep trying and share the opinions.

ywangeq commented 5 years ago

Actually, you don’t have the value for accuracy evaluation. So I am not very sure how to evaluate this.

获取 Outlook for iOShttps://aka.ms/o0ukef

发件人: jaykay notifications@github.com 发送时间: 星期二, 四月 2, 2019 11:45 上午 收件人: cguindel/eval_kitti 抄送: Yuan WANG; Mention 主题: Re: [cguindel/eval_kitti] Any examples of the format of the .txt or results generator? (#3)

Thanks for the quick reply! So, what you mean is that the score should be still filled for "3D Box regression with any value", but it doesn't actually affect evaluating 3D metrics. Is that right?

Thank you very much, I'll keep trying and share the opinions.

― You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/cguindel/eval_kitti/issues/3#issuecomment-478832948, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AYluX04-XVw46KE5QKDjNBgwn3n2HMgRks5vctJLgaJpZM4Rg0fF.

K-Young96 commented 3 years ago

@ywangeq Hello, can you share how to get the coordinate data of validation and test results?That's the format of 16 parameters data.