Questions about the optimizers used for training the sampling and fitting neural networks

[Shaluols]

Hi,

Thanks for your wonderful work! I am trying to replicate the training with the SSD dataset. Could I ask:

1. Which optimizer you used for the sampling and fitting neural network? And what are the learning rates you set for the optimizers?
2. Is it correct to use hyps, hyps_sigmas in the given test.py file to compute the loss for the sampling neural network? These two parameters are outputs from the make_graph function: means, sigmas, mixture_weights, bounded_log_sigmas, hyps, hyps_sigmas, input_blob, output_blob, tmp = session.run(output, feed_dict={x_objects: objects, x_imgs: imgs})

Thank you in advance for your help!

[os1a]

Hi @Shaluols

Thanks for your interest in our work. Here are the answers:

• We used Adam as optimizer. For the learning rate, you can find the details in our supplementary Figure 3.b (page 15) https://arxiv.org/pdf/1906.03631.pdf
• Yes, you can do that. However, notice that the make_graph returns the output of both the sampling and fitting network. So if you want to train only the sampling network, you do not need to run the fitting network. So you can stop at: https://github.com/lmb-freiburg/Multimodal-Future-Prediction/blob/d65d72f8ab80146fc3e7dac3ddd3076e2c376a94/net.py#L39.

For training the networks, sampling and fitting, you can refer to the loss functions in net.py https://github.com/lmb-freiburg/Multimodal-Future-Prediction/blob/d65d72f8ab80146fc3e7dac3ddd3076e2c376a94/net.py#L66 https://github.com/lmb-freiburg/Multimodal-Future-Prediction/blob/d65d72f8ab80146fc3e7dac3ddd3076e2c376a94/net.py#L138

Hope this helps, Best,

[Shaluols]

Thanks for your super clear reply!

[os1a]

Feel free to re-open the issue if you have more questions.