Question about "-n" and "--num_eval_episodes"

ybgdgh / L3MVN

Leveraging Large Language Models for Visual Target Navigation

https://sites.google.com/view/l3mvn

62 stars 13 forks source link

Question about "-n" and "--num_eval_episodes" #21

Open Nieysh opened 3 months ago

Nieysh commented 3 months ago

Hi! Thank you for your great work! I have some questions about evaluating episodes:

If I set "-n" to 8, "--num_eval_episodes" to 250, I get 2000 episodes to evaluate. And if I set "-n" to 4, "--num_eval_episodes" to 500, will the resulting 2000 episodes end up different with the former? Since the "--num_eval_episodes" seems to be defined for each scene.
If I want to reproduce the results in the paper, how to set the "-n" and "--num_eval_episodes"?

Nieysh commented 3 months ago

@ybgdgh The L3MVN paper says using the standard HM3D splits of 20 validation scenes, when evaluating, I notice each thread will be assigned to several scenes from the val spilts, how does each thread handle the scene and episodes data? Thanks!