Totoro97 / f2-nerf

Fast neural radiance field training with free camera trajectories
https://totoro97.github.io/projects/f2-nerf/
Apache License 2.0
933 stars 69 forks source link

Quetion about the meaning of t and dt in struct SampleResultFlex #53

Open meneldil12555 opened 1 year ago

meneldil12555 commented 1 year ago

I noticed that t and dt got used in Renderer.cpp in a way that looks like dt is the delta of t ($t_{i+1}-t_i$).

  Tensor sampled_dt = sample_result_early_stop.dt;
  Tensor sampled_t = (sample_result_early_stop.t + 1e-2f).contiguous();
  Tensor sec_density = sampled_density.index({Slc(), 0}) * sampled_dt;
  Tensor alphas = 1.f - torch::exp(-sec_density);
  Tensor idx_start_end = sample_result_early_stop.pts_idx_bounds;
  Tensor acc_density = FlexOps::AccumulateSum(sec_density, idx_start_end, false);
  Tensor trans = torch::exp(-acc_density);
  Tensor weights = trans * alphas;

But when I tried to output these two tensors, I found otherwise. The code inside PersSampler.cu suggests that dt(dists) is a somehow scaled version of the real deltas between sampled points.

sampled_ts[pts_ptr] = cur_t;
sampled_dists[pts_ptr] = exp_march_step * (proj_xyz.norm() + 1e-6f);
...
cur_t += cur_march_step;

Are those supposed behavior?

Totoro97 commented 1 year ago

Hi, yes, it is the supposed behavior. The sampled_t are the sampled distances in the original world space. The sampled_dt are the sampled distances in the warped space.