search

InhwanBae / LMTrajectory

Official Code for "Can Language Beat Numerical Regression? Language-Based Multimodal Trajectory Prediction (CVPR 2024)"
https://ihbae.com/publication/lmtrajectory/
Other
92 stars 6 forks source link

models for ETH-UCY in meter coordinates #11

Open DelinquentLeon opened 1 month ago

DelinquentLeon commented 1 month ago

Thanks for your work! I wonder if there are models pretrained for ETH-UCY in meter coordinates,cuz the models released online are in the name of xxx-pixel-multimodal-xxxx,or these can be also used for evaluating the ETH-UCY in meter coordinates?

InhwanBae commented 1 month ago

Hi @DelinquentLeon,

Thank you for your interest in our work!

The pretrained models are trained using pixel coordinates, but we project these coordinates into meter coordinates for evaluation. This way, our evaluations are directly comparable to other models that use meter coordinates. Therefore, You can directly evaluate the released models on ETH-UCY in meter coordinates, as the projection is part of the evaluation process.

https://github.com/InhwanBae/LMTrajectory/blob/4723b3943f3b8ed478b1abfa71dd0cabebbe2779/model/eval_accelerator.py#L187-L191

Let me know if you need any more clarification!

DelinquentLeon commented 1 month ago

Thanks for your kind reply! But i meet some other problems when trying to train the model for SDD dataset. I wondered how is the _reference.png created, is it selected from a random frame? Or is it matter if i choose a random frame to generate the caption for SDD dataset.

InhwanBae commented 4 weeks ago

Hi @DelinquentLeon,

For the SDD dataset, I used the same reference image provided in YNet. The _reference.png file is identical to the reference.jpg found in the annotation folder of the original SDD dataset. Since the SDD videos are static, selecting a random frame should not cause any issues in generating the caption.

DelinquentLeon commented 4 weeks ago

Thanks for your kind reply! I also wondered if you directly used the data file train_trajnet.pkl and test_trajnet.pkl from Y-Net for training and evaluating,cuz i meet some difficulties when preprocessing the SDD dataset,especially aligning the coordinates with the reference picture

DelinquentLeon commented 3 weeks ago

Additionally,could you please release the pretrained model on SDD dataset

InhwanBae commented 3 weeks ago

Hi @DelinquentLeon,

Unfortunately, I currently don't have the code and pretrained model for the SDD dataset available at the moment. Due to frequent requests, I'm considering a reimplementation it later this year. You can find more details about the SDD implementation in this issue #7.