search

MoonBlvd / tad-IROS2019

Code of the Unsupervised Traffic Accident Detection paper in Pytorch.
MIT License
164 stars 39 forks source link

Evaluation results are bad when the 'predict horizen' become longer. #3

Closed Flawless1202 closed 5 years ago

Flawless1202 commented 5 years ago

I use your code, your features and parameters to train the model, but can not get your result. Here is mine:

Model train seg length pred horizon FDE ADE FIOU
FOL 1.6 sec 0.5 sec 10.5 6.6 0.85
FOL 1.6 sec 1.0 sec 73.2 23.6 0.48
FOL 1.6 sec 1.5 sec 145.2 50.5 0.28
FOL + Ego pred 1.6 sec 0.5 sec 10.2 6.3 0.86
FOL + Ego pred 1.6 sec 1.0 sec 83.2 25.0 0.46
FOL + Ego pred 1.6 sec 1.5 sec 166.2 57.5 0.27

Is anything wrong? @MoonBlvd

MoonBlvd commented 5 years ago

@Flawless1202 Yes there are definitely errors. But it's impossible for me to tell what was wrong without knowing the details of your implementation... I trained the model with 1.0 sec prediction horizon and tested with 0.5, 1.0 and 1.5 sec. And I also trained the model with 1.5 sec prediction horizon and tested on 1.5 sec. If you trained with 0.5 sec horizon and tested for 1.0 and 1.5 sec, it's probably the reason. Also, by adding the ego motion prediction, it should be more accurate.

Flawless1202 commented 5 years ago

@MoonBlvd Thanks for your reply. I try your suggestion and get the true result!

MoonBlvd commented 5 years ago

@Flawless1202 You are welcome! I will close this issue then.