twhui
commented
2 years ago It seems to me that your ICCV 2021 paper was released to the public just a few weeks before the paper registration deadline of CVPR 2022. Therefore, I did not aware of your work when I was performing the literature review.
The technical details of RM-Depth are different from R-MSFM. Particularly, GRU is not used in RM-Depth.
Your depth CNN is basically a RAFT-based approach. Your MSFM uses GRU as RAFT to "modulate" the content between previous hidden state and encoder feature at a fixed 1/8 scale (In fact, I do not understand why you used the word "modulate" here. Maybe, "fuse" or "combine" is better).
My depth CNN uses RMU for dynamic and iterative feature fusion between encoder and decoder at several scales (Fig. 2b). Each RMU consists of two phases: modulation and update (Fig. 3). Feature from the depth encoder is modulated according to the previous hidden state through an adaptive modulation function (Eqs. 4a, 4b). Then, the new hidden state is updated as a weighted combination between the previous hidden state and the modulated encoder feature (Eqs. 5a, 5b). RMU is different from GRU. GRU does not have modulation phase. Furthermore, GRU uses two gates while RMU uses one gate.
In fact, I also proposed to use GRU for depth refinement as yours to CVPR 2021 (even earlier than yours by 4 months). However, the paper was not accepted.
jsczzzk
Congratulations on your new work!
I think the idea of iteratively refining the fusion using Recurrent Modulation Unit in RM-Depth is similar to the Recurrent Multi-Scale Feature Modulation in R-MSFM. Both of them use gated recurrent unit (GRU) to fuse the features . However, R-MSFM was not cited in RM-Depth.