Closed smittal10 closed 2 years ago
Hi @smittal10, I can think of two ways:
eval_seq2seq.py
on a small portion of validation tasks that captures a good distribution of tasks in the dataset (clean, pick etc.)Hope this helps!
Closing due to inactivity.
Hi, is there a way to evaluate the seq2seq model checkpoint using a static test dataset that doesn't require online interactions with the simulator? Currently eval_seq2seq.py evaluates it interactively with the simulator. We want to run inference on a small device with limited compute resources and running the simulator there is not feasible.