dosubot[bot]
commented
7 months ago Hey there, @fekstr! Nice to see you diving into the wonders of 3D geometries with us again. 😊
The part-affordance-learning repository is designed to facilitate the learning of affordances of object parts based on their 3D geometry. This is achieved through a series of steps that involve preprocessing 3D object meshes from the PartNet database, creating labeled point clouds from these meshes, configuring training and testing parameters, and finally, training and testing a model to learn these affordances.
The process involves:
- Downloading 3D object meshes from PartNet, a comprehensive database of 3D objects.
- Merging object meshes into full parts to create a unified representation of each object.
- Creating point clouds with labels from the merged meshes, which serve as the input data for the model. These point clouds are annotated with information about the parts of the objects they represent.
- Configuring the training and testing environment by setting object classes, hyperparameters, and logging details.
- Training a model to learn the affordances of object parts from the labeled point clouds. This involves setting up a specific model architecture and loss function, and saving checkpoints for later evaluation.
- Testing the trained model using a specified checkpoint and visualizing the results to assess the model's ability to accurately learn and predict the affordances of object parts based on their 3D geometry.
The application of this repository is in the field of robotics and computer vision, where understanding the affordances of object parts (i.e., what actions can be performed with them) is crucial for tasks such as object manipulation, interaction, and navigation in environments populated with objects. You can find more details in the README.md file.
The specific 3D object meshes used from the PartNet database in the part-affordance-learning repository are Chair, Table, Microwave, Dishwasher, Backpack, Briefcase, Mug, Jug, Normal Bottle, Bed, Knife, Scissors, and Door Set. This information can be found in the scripts/preprocessing/manual_part_labels.json file.
The specific model architecture used in the part-affordance-learning repository for learning the affordances of object parts combines several components: PointNet2SemMsg, MultiheadAttention, AffordanceClassifier, Segmenter, and AttentionModel. This architecture is designed to learn the affordances of object parts by leveraging point cloud data, attention mechanisms, and deep learning techniques for classification and segmentation tasks. More details can be found in the src/models/attention.py file.
fekstr
What is this repo for?