Is the ARAP Constraint in SpatialTracker disabled?

[Jafonso-sudo]

Hi SpatialTracker team,

First of all, thank you for the fantastic work on this project! I’m currently exploring SpatialTracker to track rigid objects and was particularly interested in applying the ARAP (As-Rigid-As-Possible) constraint. Specifically, I was trying to "hack" the ARAP constraint by forcing all queried points to have the same ARAP embedding.

However, in the process, I noticed that the ARAP constraint might not be active in the current main commit. I’m hoping to clarify whether my understanding is correct. Here's why I believe ARAP might be disabled in the model:

What I’ve Found:

1. __init__ Function:

   • There’s an argument for enabling/disabling ARAP (self.args.if_ARAP), but the only place where this is referenced is in the forward_iteration function:
   • Interestingly, the ARAP-related code in the forward_iteration function is commented out:

2. neural_arap Function:

   • When searching for references to the neural_arap function, it appears this function isn't called anywhere else in the code:
   • I also tested commenting it out, and the model still runs without issues.

3. self.embed_traj Initialization:

   • There’s a TODO in the __init__ function to optimize ARAP, which further suggests that ARAP might not be fully implemented:
   • The variable self.embed_traj (initialized here) is only used within the neural_arap function, which as mentioned, isn’t called anywhere else, so this initialization can also be commented out without breaking the model.

4. Traj_arap in forward:

   • In the forward function, the variable Traj_arap, returned by self.prepare_tracks, is declared but never used (as it appears greyed out in VSCode):
   • Additionally, within self.prepare_tracks, the Traj_arap doesn’t seem to influence anything further in the code.

My Question:

I might be misunderstanding how ARAP is intended to be used here, but from what I can tell, the ARAP embeddings don’t seem to have an effect. Is it possible that ARAP isn’t fully implemented or has been temporarily disabled? Or could there be an updated commit that hasn't yet been pushed to this repository?

ARAP seems like a very exciting feature, and it's core to the application I'm trying to use SpatialTracker for, so I'd love to understand its current status.

Thanks so much for any clarification or insights you can provide! I really appreciate your time, and I’m eager to continue working with SpatialTracker.

Best regards,
João Mendonça

[Jafonso-sudo]

I think I now understand that the ARAP embedding is only used during training to come up with the ARAP loss, but not directly during inference as well. I'm still not sure due to the commented out code saying: "refine the track with arap".