search

cunjunyu / STAR

[ECCV 2020] Code for "Spatio-Temporal Graph Transformer Networks for Pedestrian Trajectory Prediction"
MIT License
366 stars 82 forks source link

How can we deal with dynamic number of human? #16

Open mincheulkim opened 3 years ago

mincheulkim commented 3 years ago

Hi, I'm very impressed with your work. I'd like to know your thoughts on the following issues related to human trajectory prediction on training real datasets.

  1. What if the number of people changes at each timestamps? e.g., the number of people per scene may varies(5⇒5⇒6⇒8⇒1⇒...)
  2. Even if the number of people is fixed for each timestamp, what if the people who make up the scene change? e.g. at t=1, human id=1,2,3,4,5 ⇒ t=2, human id=2,3,4,6,7(human 1,5 are missing but human 6,7 are pop in).

The above problems may not occur in the simulation using a synthetic dataset in which the number of people is unchanged, but it may be happen on the train with real datasets. I wish to hear your opinion on this?

Thanks,

a79687417 commented 3 years ago

Hi, I'm doing a trajectory prediction research as well, my reseach is mainly based on Social GAN (SGAN), but I think most of the LSTM based methods treat the trajectory/position input pretty much the same way. I might be wrong though, please correct me if I'm wrong.

In most of my referenced methods, they only consider trajectories in same timestamps, for example, if we're using 8 observed length trajectory input, when considering interaction relations at t=0, we'd only compute all trajectories that exist from t=0, t=1, ... to t=7. If a certain person with trajectory only exists from t=0 to t=4, or from t=0 to t=7 but missing a frame between, then this trajectory would not be processed.

In the missing frame case, some would use interpolating method to fix it, there are some examples in adjusted ETH/UCY dataset like: https://github.com/erichhhhho/DataExtraction , but if someone leave the scene with trajectory insufficient to the set-up length, then his/her trajectory won't be processed. Another way to fix this may be reducing your observed length and prediction length, or even increace fps of your samples from datasets.

cunjunyu commented 3 years ago

Sorry for the late reply.

Yes, @a79687417 is right about this (Thank you for helping answer the question).

Most of the methods only consider trajectories in the same timestamps. Trajectories shorter than predefined are not considered.

xiaocc612 commented 2 years ago

Can anyone tell me how these remaining tracks are generally handled? Thanks.