prophesee-ai / prophesee-automotive-dataset-toolbox

A set of Python scripts to evaluate the Automotive Datasets provided by Prophesee
Apache License 2.0
153 stars 30 forks source link

Some annotations for labels on GEN1 dataset are wrong #41

Open JDYG opened 1 week ago

JDYG commented 1 week ago

The ground truth annotation for ./Gen1_Automotive/test/17-10-12_16-51-41_2989500000_3049500000_td.dat in GEN1 dataset is wrong.

The code for loading this label is:

event_path ='./Gen1_Automotive/test/17-10-12_16-51-41_2989500000_3049500000_td.dat'
bbox_fname = event_path [:-7] + '_bbox.npy'
label_path= PSEELoader(event_path)
box_video = PSEELoader(label_path)

total_time = events_video.total_time()
events = events_video.load_delta_t(total_time)
labels = box_video.load_delta_t(total_time)

The extracted results for labels are:

[(10849999, 255., 91., 52., 38., 0, 1., 3765),
       (11849999, 255., 91., 52., 38., 0, 1., 3766),
       (12849999, 255., 91., 52., 38., 0, 1., 3767),
       (13849999, 255., 91., 52., 38., 0, 1., 3768),
       (14849999, 255., 91., 52., 38., 0, 1., 3769),
       (15849999, 255., 91., 52., 38., 0, 1., 3770),
       (16849999, 255., 91., 52., 38., 0, 1., 3771),
       (17849999, 255., 91., 52., 38., 0, 1., 3772),
       (18849999, 255., 91., 52., 38., 0, 1., 3773),
       (19849999, 255., 91., 52., 38., 0, 1., 3774),
       (20849999, 255., 91., 52., 38., 0, 1., 3775),
       (21849999, 255., 91., 52., 38., 0, 1., 3776),
       (22849999, 255., 91., 52., 38., 0, 1., 3777),
       (23849999, 255., 91., 52., 38., 0, 1., 3778),
       (24849999, 255., 91., 52., 38., 0, 1., 3779),
       (25849999, 255., 91., 52., 38., 0, 1., 3780),
       (26849999, 255., 91., 52., 38., 0, 1., 3781),
       (27849999, 255., 91., 52., 38., 0, 1., 3782),
       (28849999, 255., 91., 52., 38., 0, 1., 3783),
       (29849999, 255., 91., 52., 38., 0, 1., 3784),
       (30849999, 255., 91., 52., 38., 0, 1., 3785),
       (31849999, 255., 91., 52., 38., 0, 1., 3786),
       (32849999, 255., 91., 52., 38., 0, 1., 3787),
       (33849999, 255., 91., 52., 38., 0, 1., 3788),
       (34849999, 255., 91., 52., 38., 0, 1., 3789),
       (35849999, 255., 91., 55., 41., 0, 1., 3790),
       (36849999, 255., 90., 55., 38., 0, 1., 3791),
       (37849999, 256., 91., 54., 37., 0, 1., 3792),
       (38849999, 254., 89., 56., 43., 0, 1., 3793),
       (39849999, 255., 90., 52., 42., 0, 1., 3794),
       (40849999, 255., 89., 50., 43., 0, 1., 3795),
       (41849999, 255., 90., 55., 42., 0, 1., 3796),
       (42849999, 255., 91., 55., 41., 0, 1., 3797),
.....

In a 2D visualization, the gt box is always on the same location and is incorrect, as shown below: image

image

lbristiel-psee commented 5 days ago

Hello,

Thank you for bringing this to our attention. It’s possible that some of the files may indeed have issues. Have you found the annotations to be correct in most of the other files?

Best regards, Laurent

JDYG commented 5 days ago

All other annotations are correct, except for this one. I have only checked part of the dataset, not the entire dataset. If I found other uncorrect label, I will report here.