georghess / neurad-studio

[CVPR2024] NeuRAD: Neural Rendering for Autonomous Driving
https://research.zenseact.com/publications/neurad/
Apache License 2.0
346 stars 24 forks source link

Error when trying to remove actors + fixing solution #42

Open Andyshen555 opened 4 months ago

Andyshen555 commented 4 months ago

Describe the bug When I try to remove all actors during rendering, throws IndexError: shape mismatch.

To Reproduce Steps to reproduce the behavior:

  1. Train on the pandaset as described
  2. Render with the option --actor-removal-time x.x
  3. See error (for sequence 001 and actor removal time 0.0)
    nerfstudio/scripts/render.py", line 739, in modify_actors
    pipeline.model.dynamic_actors.actor_present_at_time[no_actor_mask, indices] = False
    IndexError: shape mismatch: indexing tensors could not be broadcast together with shapes [40], [47]

Expected behavior the no_actor_mask and indices should be used as indices instead of concatenating together.

Additional context As far as pytorch 2.0.1-2.3.1, the > operator outputs tensor [true, false ,....] instead of index. I solved the problem by replacing line 738 with no_actor_mask = torch.nonzero(pipeline.model.dynamic_actors.unique_timestamps > actor_removal_time, as_tuple=False)