search

XT5un / P2NeRF

[CVPR2024] Global and Hierarchical Geometry Consistency Priors for Few-shot NeRFs in Indoor Scenes
10 stars 1 forks source link

question about keypoint prior of custom dataset #2

Closed zhanghaoyu816 closed 3 weeks ago

zhanghaoyu816 commented 1 month ago

Thanks for your nice work! I would like to validate your method on other indoor datasets, but I'm not sure how to get the keypoint-pairs prior of dataset. You mentioned in your paper that you used LoFTR to extract feature point matches and did some other post-processing, but I'm still not sure of the process. So could you please release the code of generating prior so that I can test your method on some other dataset. Thanks a lot!

zhanghaoyu816 commented 1 month ago

When I attempted to replicate your method on DDP dataset, I encountered several potential bugs and some questions, which I have listed here for reference.

  1. Does the depth_weight_mask_margin defined in L147 refers Config.depth_rank_margin ? If so, they should be the same name.
  2. In L964-L967, self.images is being used before it is declared, which leads to an error.
  3. I want to know what is Ellipsis? It seems not cause a program error, but it is highlighted in red as a warning in VSCode图片
  4. I am curious about the memory usage. In your paper, you mentioned that you train your method on a single RTX3090 with batchsize 4096. When I replicated your method without change the experimental settings in the config file(Config.batch_size = 4096), I found that the memory usage has exceeded 40 GB！Is this a reasonable situation?图片
XT5un commented 1 month ago

I've uploaded the code get_kpts_prior.py that generates the keypoint prior, and you can follow the instructions to generate the relevant data.

1 & 2: The related bug has been fixed(1ea44535befb8a0a3ffe783a8f48f20f4be0c98c), thanks for the heads up.

3: This is the code in the original RegNeRF, and removing it will have no effect.

4: Jax's mechanism is to pre-allocate more GPU memory, if you don't need it to pre-allocate, you can refer to the official documentation to set it up.