siddk / voltron-evaluation

Voltron Evaluation: Diverse Evaluation Tasks for Robotic Representation Learning
MIT License
32 stars 10 forks source link

[Roadmap] Consolidate V-Evaluation Harness into a General API / Runner #4

Open siddk opened 1 year ago

siddk commented 1 year ago

Once the Visuomotor Control & Intent Scoring tasks have been integrated properly, would be nice to consolidate the V-Evaluation Harness into a more general API that can be used for other downstream tasks.

Would be nice to also allow for a programmatic "runner" – specify a single backbone/extraction mechanisms, then automatically run all tasks with a single script.