Aria Training and Evaluation toolkit (ATEK)

https://github.com/user-attachments/assets/a3613d17-d272-471e-9e9f-fffb23b1990d

The Aria Training and Evaluation Kit (ATEK) is a toolbox for accelerating the development of Machine Learning tasks using Aria datasets. Specifically, it provides

• An easy-to-use data preprocessing library for Aria datasets for converting VRS-format Aria datasets in to datasets in PyTorch-compatible format
• Datastore of preprocessed data sets in WebDataset format
• Standardized evaluation libraries that supports the following ML perception tasks for Aria:
  • static 3D object detection
  • 3D surface reconstruction
• Notebooks and script examples including model training, inference, and visualization

ATEK users can engage ATEK in their projects with 3 different starting points:

• Just want to run some evaluation, even on non-Aria data? Check out ATEK evaluation libraries!
• Want to try your trained-model on ego-centric Aria data? Just download processed data from our Data Store, and check out how to run model inference!
• Now ready for the full ML adventure from raw Aria data? Check out our full table of contents!

Interactive Python notebook playground (Google Colab)

User can start with our Google Colab notebook, which shows an example of running and evaluating a 3D object detection model called CubeRCNN, on an Aria Digital Twin data sequence, which involves data-preprocessing, model inference, and evaluation.

Table of content

• Installation

• ATEK Data Store

• Technical specifications

  • Data Preprocessing
  • Data Loader for inference and training
  • Model Adaptors
  • Evaluation

• [Machine Learning tasks supported by ATEK]

  • static 3D object detection
  • 3D surface reconstruction

• Examples

  • Example: demo notebooks
  • Example: customization for SegmentAnything2 model
  • Example: customization for CubeRCNN model
  • Example: CubeRCNN model training

License