Analytical Gradient Coarse-to-Fine Training

[justachetan]

Hi @mli0603 ,

In commit https://github.com/NVlabs/neuralangelo/commit/deb9b4562c42637ad1f3ef0af25c57748d8265e2 , it seems that coarse-to-fine training has been removed for analytical gradients (since set_active_levels is never called to set active_levels for analytical gradient mode). Is this intentional behavior?

I am running into an error when running the analytical gradient version of the pipeline with the current version of the codebase:

Traceback (most recent call last):
  File "train.py", line 104, in <module>
    main()
  File "train.py", line 93, in main
    trainer.train(cfg,
  File "/home/ac2538/work/neuralangelo/projects/neuralangelo/trainer.py", line 107, in train
    super().train(cfg, data_loader, single_gpu, profile, show_pbar)
  File "/home/ac2538/work/neuralangelo/projects/nerf/trainers/base.py", line 109, in train
    data_all = self.test(self.eval_data_loader, mode="val", show_pbar=show_pbar)
  File "/home/ac2538/.conda/envs/neuralangelo/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/ac2538/work/neuralangelo/projects/nerf/trainers/base.py", line 138, in test
    output = model.inference(data)
  File "/home/ac2538/.conda/envs/neuralangelo/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/ac2538/work/neuralangelo/projects/neuralangelo/model.py", line 77, in inference
    output = self.render_image(data["pose"], data["intr"], image_size=self.image_size_val,
  File "/home/ac2538/work/neuralangelo/projects/neuralangelo/model.py", line 103, in render_image
    output_batch = self.render_rays(center, ray_unit, sample_idx=sample_idx, stratified=stratified)
  File "/home/ac2538/work/neuralangelo/projects/neuralangelo/model.py", line 128, in render_rays
    output_object = self.render_rays_object(center, ray_unit, near, far, outside, app, stratified=stratified)
  File "/home/ac2538/work/neuralangelo/projects/neuralangelo/model.py", line 160, in render_rays_object
    dists = self.sample_dists_all(center, ray_unit, near, far, stratified=stratified)  # [B,R,N,3]
  File "/home/ac2538/.conda/envs/neuralangelo/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/ac2538/work/neuralangelo/projects/neuralangelo/model.py", line 238, in sample_dists_all
    sdfs = self.neural_sdf.sdf(points)  # [B,R,N]
  File "/home/ac2538/work/neuralangelo/projects/neuralangelo/utils/modules.py", line 75, in sdf
    return self.forward(points_3D, with_sdf=True, with_feat=False)[0]
  File "/home/ac2538/work/neuralangelo/projects/neuralangelo/utils/modules.py", line 70, in forward
    points_enc = self.encode(points_3D)  # [...,3+LD]
  File "/home/ac2538/work/neuralangelo/projects/neuralangelo/utils/modules.py", line 93, in encode
    mask = self._get_coarse2fine_mask(points_enc, feat_dim=feat_dim)
  File "/home/ac2538/.conda/envs/neuralangelo/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
  File "/home/ac2538/work/neuralangelo/projects/neuralangelo/utils/modules.py", line 115, in _get_coarse2fine_mask
    mask[..., :(self.active_levels * feat_dim)] = 1
  File "/home/ac2538/.conda/envs/neuralangelo/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1614, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'NeuralSDF' object has no attribute 'active_levels'

I have followed your advice for launching the analytical gradient variant of the pipeline described in the discussion in #22

Any advice on how to proceed would be much appreciated. Thanks!

[mli0603]

Hi @justachetan

Can you see if https://github.com/NVlabs/neuralangelo/commit/66808b9603b56462914300e4a56e2e1dcd154934 fixes your issue?

[justachetan]

Trying it out. Will keep you posted.

[justachetan]

Hi, yes this fixes the issue. Though surprisingly I am seeing better results for no coarse-to-fine training as opposed to the one where coarse-to-fine training is enabled.

[mli0603]

Hi @justachetan

Glad to know the issues have been fixed!

• The differences are small and it is expected that sometimes with c2f the model converges slower. If you wait for the entire training to complete, you may see different final results.
• Also, this PSNR logged in W&B is only a subset of the images. There may be some differences if you look at the entire data.

[justachetan]

Got it, thanks!