Usage of ray_resampling function

[WenhongZhang]

Dear nerfacc developers, I noticed an utils function 'nerfacc.ray_resampling' in document, but it is not compatible with 'nerfacc.ray_marching' (0.3.4 version) since it requires 'packed_info' https://github.com/KAIR-BAIR/nerfacc/blob/cf7ebaf168a3932977e31b258e050c6df0fdd43a/nerfacc/cdf.py#L13 . Resampling strategy is quite important, and I wonder if there is way to do so. Thanks in advance!

[liruilong940607]

packed_info can be created from ray_indices: https://github.com/KAIR-BAIR/nerfacc/blob/cf7ebaf168a3932977e31b258e050c6df0fdd43a/nerfacc/pack.py#L47-L48

I just realized it is not on the doc. You can just import it from nerfacc. Will update later!

[WenhongZhang]

packed_info can be created from ray_indices:

https://github.com/KAIR-BAIR/nerfacc/blob/cf7ebaf168a3932977e31b258e050c6df0fdd43a/nerfacc/pack.py#L47-L48

I just realized it is not on the doc. You can just import it from nerfacc. Will update later!

It works, but how to combine coarse-samples from ray_marching and fine-samples from ray_resampling? Could I just simply concat ray_indices/t_starts/t_ends as followed?

alphas = alpha_fn(t_starts_coarse, t_ends_coarse, ray_indices_coarse)
weights = render_weight_from_alpha(alphas, ray_indices=ray_indices_coarse, n_rays=rays.shape[0])
packed_info = pack_info(ray_indices=ray_indices_coarse, n_rays=rays.shape[0])
packed_info_fine, t_starts_fine, t_ends_fine = ray_resampling(packed_info, t_starts_coarse, t_ends_coarse, weights[:, 0], 16)
ray_indices_fine = unpack_info(packed_info_fine, t_starts_fine.shape[0])

ray_indices = torch.cat([ray_indices_coarse, ray_indices_fine])
t_starts = torch.cat([t_starts_coarse, t_starts_fine])
t_ends = torch.cat([t_ends_coarse, t_ends_fine])

[liruilong940607]

No that would not work.

Unfortunately currently combining the samples are not supported. It would require some cuda coding.

[WenhongZhang]

Thanks for your reply. I'll implement that somehow.