search

zjjMaiMai / TinyHITNet

HITNet: Hierarchical Iterative Tile Refinement Network for Real-time Stereo Matching
152 stars 21 forks source link

Question about the difference between HitNet model for SF and for KITTI. #19

Open dzngo opened 1 year ago

dzngo commented 1 year ago

First of all, I would like say that your work is amazing!!! It has helped me a lot!!! Thank you!

However, I have question about the difference between Hitnet_SF and Hitnet_KITTI. I see that the Hitnet_SF doesn't give the output of hypothesis confidence (w), so that during training, the loss for w won't be taken into account. Moreover, it seems that the Initialization part and Propagation part are also different between 2 models in your code. I see in the supplementary material of original paper, the authors say that the difference between models for each dataset is simply channel size of feature extractor.

I saw that your response for a closed issue that you succeeded on converting the original model weight, which is in format tensorflow buffer provided by the authors, into the pytorch model and obtain almost the same result on SF final_pass. It would be so grateful for me if you would share how did you interpret the tensorflow pb model to rewrite the entire model in pytorch.

Thank you again for your work and look forward for your response!

zjjMaiMai commented 10 months ago

Hi @dzngo, sorry for the late reply.

Hitnet_SF and Hitnet_KITTI model structure are exactly same with origin tensorflow checkpoints. so I think there is some difference here between the author's implementation and the paper description.

for the weights convert from tensorflow to pytorch, i convert them one by one manually. and i write some unit tests to ensure the correctness of numerical conversions.