search

ClementPinard / SfmLearner-Pytorch

Pytorch version of SfmLearner from Tinghui Zhou et al.
MIT License
1.01k stars 224 forks source link

Testing pose with 3 frame snippets #88

Closed raunaks13 closed 4 years ago

raunaks13 commented 4 years ago

I used your code to compute absolute trajectory error (ATE) for a sequence length of 3. Since the standard for most papers is a sequence length of 5, is it still fair to compare the ATE with that of other models?

ClementPinard commented 4 years ago

No, 3 frame snippets are easier to compute. ATE is not really ATE because every pose is written relative to the first Ground Truth pose. As of now, the pose estimation code is flawed (but I copied it from the original implementation to be more consistent), every pose is perfect and counted is the evanuation, and translation is rescaled to have the same total magnitude than GT.

raunaks13 commented 4 years ago

I have 2 questions:

  1. Even though it may be computationally faster to compute it for 3 frame snippets, the error isn't fundamentally lesser compared to 5 frame snippets, right? So comparing the two purely based on magnitude of error should be fine?
  2. So even though the ATE calculation may be flawed, it matches the way the original GeoNet paper computed the ATE, right?
raunaks13 commented 4 years ago

Any ideas @ClementPinard ?

ClementPinard commented 4 years ago
  1. The longer the frame snippet, the greater the error, because ATE takes its name from the fact that you compare the absolute position, meaning every displacement estimation error gets added up. Here, trajectory is not really absolute because everything it relative to the first frame, the longer the snippet gets, the more it becomes the same as an actual ATE.
  2. Yes, it's the same technique