search

haleqiu / AirIMU

BSD 3-Clause "New" or "Revised" License
45 stars 10 forks source link

Questions on the results #3

Closed Endless-Horizon closed 2 months ago

Endless-Horizon commented 4 months ago

Hello, I have a question, I saw that you compared Baseline and Zhang's method in your paper and I would like to know why Baseline performs better than Zhang's method? As I understand, Baseline doesn't make any corrections to the IMU measurements.

haleqiu commented 4 months ago

@Endless-Horizon Since the code and model from Zhang et al. have not been released, I can only compare them with the results they posted on the paper.

However, I can share some of my insights on why the results are different from Zhang et al.. First, we build our own differentiable IMU integrator on manifolds. You may check the details on PyPose. Second, I think the end-to-to neural network introduces ambiguity to the output IMU. It may average the IMU inputs and "round" to the label. Even Zhang et al. propose a loss to alleviate it. From my point of view, this problem may not be fully addressed. Especially when integrating small segments like 10 frames. This ambiguity will be magnified.

haleqiu commented 4 months ago

We will release an extended paper this week. Hope you can learn more about the insight from this version

Endless-Horizon commented 4 months ago

@haleqiu Thank you very much for your reply! But I still have a few questions. First of all, I also read zhang's paper, and from my point of view, zhang's idea is also to utilize DNN to get the refined measurements, and then train them by IMU integration. I think this is not an end-to-end approach. In zhang's paper, he also compared several methods, including a Traditional method (termed Tr.), and the result was that zhang's method showed the best performance. From my understanding, the Traditional method should be close to the pure IMU integral (Baseline). I don't know if I misunderstood this, hope you can answer. Secondly, I got the same result by calculating Baseline through your code, but I still have trouble understanding why the methods proposed by these scholars do not perform as well as Baseline. From my point of view, Baseline is a differentiable IMU integrator that you have integrated on python, and it is still based on the derivation of the IMU integration without any other optimization. I'm wondering if the metrics are calculated differently? I noticed that zhang's paper calculates the "relative" pose accuracy. I'm looking forward to reading your extended paper!

haleqiu commented 4 months ago

@Endless-Horizon Hi,

I think the term "end-to-end" I used may cause some misunderstandings. Zhang et al.'s model uses the neural network to directly regress the corrected acceleration and angular velocity for the integration. For example, if the network learns a way to average the acceleration and outputs the averaged acceleration, the integration result will be the same as the non-averaged acceleration for long segments. However, this may introduce considerable errors in a short period of integration. Think about how we supervised the model; you'll understand how this happened. That's part of the reason why we prefer the data-driven method to predict the IMU corrections, not the measurement itself.

Regarding the evaluation benchmark, they show the RMSE for ten frames of integration. We also show the RMSE for 10-frame integration. Ideally, the integrator should not drift a lot in 10 frames. Since the code has yet to be released, there is little thing we can do.

Hope this can answer your question

Endless-Horizon commented 4 months ago

Thank you very much for your reply!