Missing Files in Google Drive

[Yang-Li-2000]

I cannot find some of the files when running the validation scripts for marginal trajectory prediction and conditional trajectory prediction. Specifically, checkpoint paths such as densetnt.raster.vehicle.1 and densetnt.reactor.Tgt-Rgt.raster_inf.v2v are in the scripts but are not on the provided Google Drive page.

I tried to use checkpoints whose names are most similar to the ones in the scripts but using these checkpoints results in missing keys and size mismatches. Additionally, from the Google Drive page, I cannot find specified RELATION_PRED_DIR and INFLUENCER_PRED_DIR, whose names match those mentioned in the scripts, either.

Could you please provide the missing files and more detailed instructions for running the validation scripts?

[larksq]

For checkpoint zip files, I updated the file names in the readme with the same file names on Google Drive. Could you please try to load again as instructed? Let me know if you still have a missing-keys problem. And RELATION_PRED_DIR and INFLUENCER_PRED_DIR are the paths of the prediction result pickle files.

[Yang-Li-2000]

Hi, I used the evaluation scripts in the updated readme for Conditional Trajectory Prediction but I am still having the missing keys and size mismatch problems. Specifically, the default value of validation_model in conditional_pred.yaml is 30, but there is not a checkpoint for the 30th epoch. As a result, I changed 30 to 29. However, when using the checkpoint for the 29th epoch, I am still encountering missing keys and size mismatch problems.

[larksq]

Seems like there is a channel size mismatch. Could you try to delete the 'raster_inf' flag in 'other_params' from the conditional_pred.yaml file and try it again. Thanks.

[Yang-Li-2000]

Hi Alan, I removed raster_inf from conditional_pred.yaml but the missing keys and size mismatch problems still exist.

[larksq]

I found the problem. It should be 'train_reactor' instead of 'train_relation' in the conditional_pred.yaml file. Could you please update the conditional_pred.yaml file and give it another try? It should work this time.

[Yang-Li-2000]

Hi Alan, thank you. Changing train_relation to train_reactor works. However, after saving merged result, the function motion_metrics.result() throws an assertion error.

[larksq]

You can ignore this error and inspect the prediction result saved in the pickle file. It would be more complicated if you want to evaluate the conditional prediction result properly. To evaluate prediction results with Waymo's interactive dataset API, the shape for each scenario needs to be (6, n, 80, 2). The result needs 6 different predictions and n is the number of agents to predict (n=2 for the interactive dataset). You would need to merge 6 groups of prediction results of the reactors by a conditional prediction for each influencer prediction. Unfortunately, this part of code is not released in this repo.

[Yang-Li-2000]

Hi Alan,

1. I am not sure which 6 groups should I merge. When running the evaluation command, it generates 6 predicted trajectories for each scenario. However, in result() in class MotionMetrics in src/waymo_tutorial.py, only 1 trajectory is left for each scenario. The shape of prediction_trajectory in result() in class MotionMetrics is [num_scenarios, 80, 2].

   

At the same time, in REDAME.md, --eval_rst_saving_number can have 6 different values (from 0 to 5).

Should I merge 6 predicted trajectories for a fixed eval_rst_saving_number or merge predicted trajectories generated using 6 different eval_rst_saving_number?

2. Additionally, when merging the trajectory of the influencer and the trajectory of the reactor, how should I permute them? The shape of the merged trajectories is (num_scenarios, 6, n=2, 80, 2). Is [:, :, 0, :] for influencers and [:, :, 1, :] for reactors? Or, is there a different way to arrange influencer trajectory and reactor trajectory?

3. How should I merge the scores of the influencer and the scores of the reactor? Should I multiply them or stack them?

[hangzhaomit]

I would suggest opening another issue if the original question is resolved.