Evaluation protocol for local exp

[Zed-Wu]

I found for some environments like PickSingleYCB, the evaluation protocol in the documents shows that there are not only 5 episodes per object in the training set (74 YCB objects) but also 10 episodes per object in the held-out evaluation set (40 objects from other sources). Are the objects in the held-out evaluation set (40 objects from other sources) only used in the ManiSkill2 Challenge? or can I evaluate them locally?

[xuanlinli17]

Sorry for the late reply. We currently don't have plans to release the held-out evaluation set yet. They are currently only used for the MS2 challenge.