search

megvii-research / Sobolev_INRs

[ECCV 2022] The official experimental code of "Sobolev Training for Implicit Neural Representations with Approximated Image Derivatives"
MIT License
30 stars 2 forks source link

Problems on other NeRF models #2

Open C0SM00 opened 1 year ago

C0SM00 commented 1 year ago

I tried to reproduce your gradient supervision method on the Tensorf model, but the calculated rgb-to-coordinate gradient is not in the same order of magnitude as the target gradient. The former is around 1e-6, and the latter is around 1e-1. Excuse me. Is there a problem with reproduction?

wtyuan96 commented 1 year ago

Thank you for your interest in our work.

We have not test our method with TensoRF, but you can try from the following aspects:

  1. check the the input coordinate of the function diff_operators.gradient, the coordinate should not be normalized, i.e. the coordinate should be the same as when calculating the target gradient.
  2. calculate the rgb-to-coordinate gradient after the TensoRF model optimization (without supervision on derivatives) is complete, not when the model is initialized.
  3. Some other factors that may have an impact: use LLFF dataset, remove view direction dependency.

Good luck!

C0SM00 commented 1 year ago

I ran your original code with the command line python train.py --config configs/fern/val_der/sine.txt, but why didn't I observe a drop in der_loss?

wtyuan96 commented 1 year ago

Maybe you can check your reproduction as I said first. By the way, TinyNeRF and TensoRF are different implementations of INRs, i.e. frequency encoding and parametric encoding (from Instant-NGP), the activation function and the approximation capability of derivative may also have an impact.