search

umautobots / bidirection-trajectory-predicter

The code for Bi-directional Trajectory Prediction (BiTraP).
Other
78 stars 23 forks source link

Questions for Preprocessing #15

Open tcheung99 opened 2 years ago

tcheung99 commented 2 years ago

Hi @MoonBlvd,

I am trying to train the BiTraP model on the nuScenes dataset and replicate the results stated in section III of the Supplementary File in your paper. I have a few questions on the setup used for nuScenes, since my reproduction results are different from the 0.58 FDE reported in the paper.

  1. How was the nuScenes dataset preprocessed for pedestrian-only? (We used the preprocessing code from Trajectron++, and we removed code that references vehicle to produce a pedestrian-only nuScenes pickle).
  2. How many frames are used for observation? (We used 8 frames for observation and 8 frames for prediction).

Another issue is that our training results and testing results are drastically different. Following the instructions in this repo's README, we used the files under https://github.com/umautobots/bidireaction-trajectory-prediction/tree/main/tools for training and testing. For training, we saw the FDE drop to around 0.39, but when running test.py, the FDE was around 2.07. Have you run into this issue before?

Thanks in advance!

MoonBlvd commented 2 years ago

Hi @tcheung99 I'm sorry for the late response.

How was the nuScenes dataset preprocessed for pedestrian-only?

I believe we used the exact preprocssing used by Trajectron++. Unfortunately I graduated 2 years ago and cannot access the computer and the original configs I used for the nuScenes experiments. Can you please how me your config file used for nuScenes training? And also what was your FDE results?

How many frames are used for observation? (We used 8 frames for observation and 8 frames for prediction).

I think we used the same observation length and prediction horizon with trajectron++. I think nuScenes is 2Hz, so it should be 8 frames, the same to your setting.

For training, we saw the FDE drop to around 0.39, but when running test.py, the FDE was around 2.07. Have you run into this issue before?

No I haven't seen this before. Have you tried testing our provided ckpts for ETH-UCY? Does it also have large discrepancy between train and eval?