how to export contact info

[MandiZhao]

How can I generate & export the per-step hand-object binary contact information? I see the scripts_method/visualize.py has an option to visualize it from model predictions, but I'm trying to generate GT contact from the raw data. Thanks in advance!

[zc-alexfan]

Hi @MandiZhao, we have a task called "interaction field estimation" in the paper, which is a fully-supervised method to estimate hand-object distances/contacts.

You can find its training script here: https://github.com/zc-alexfan/arctic/blob/master/scripts_method/train.py where we prepared the targets for the model.

[MandiZhao]

train.py does not contain how target data was prepared. I saw the dataset is constructed it's somewhere in src/datasets/arctic_dataset.py, but I still can't find keys like target.dist.or in ArcticDataset.getitem anywhere, while these dist keys are required in construct_mesh from src/mesh_loaders/field.py -- is it possible some part of the code did not get released?

[zc-alexfan]

https://github.com/zc-alexfan/arctic/blob/master/src/callbacks/process/process_generic.py

[zc-alexfan]

https://github.com/zc-alexfan/arctic/blob/master/src/callbacks/process/process_field.py

[MandiZhao]

@zc-alexfan unfortunately the links alone were not very helpful. What I found after much digging around is that, I needed to get the global hand and object vertices per time-step, which can be found as from processed_data (processed_data['world_coord']['verts.right'] etc) after running process_seq.py on raw_seqs, and combine them with the hand/object mesh faces. Distance/contact field can then be extracted using knn on the vertices. Closing this issue for now.

[zc-alexfan]

Hi @MandiZhao , sorry I was on the bus when replying to the issue. Yes your overall approach is the same way that I used. This link shows how to use the knn to get the distance values for contact. Hope it clarifies.