a-lemus96 / fs-nerf

PyTorch implementation for experimenting with frequency regularized Neural Radiance Fields.
0 stars 0 forks source link

Mismatch between devices in `get_rays` fn #73

Closed a-lemus96 closed 3 months ago

a-lemus96 commented 3 months ago
Traceback (most recent call last):
  File "/home/lemus/projects/fs-nerf/src/run-nerf.py", line 540, in <module>
    main()
  File "/home/lemus/projects/fs-nerf/src/run-nerf.py", line 467, in main
    val_metrics = validation(
                  ^^^^^^^^^^^
  File "/home/lemus/projects/fs-nerf/src/run-nerf.py", line 132, in validation
    rgbs, _ = renderer.render_poses((H, W, focal),
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lemus/projects/fs-nerf/src/render/renderer.py", line 136, in render_poses
    rays_o, rays_d = utils.get_rays(H, W, focal, poses, device)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/lemus/projects/fs-nerf/src/utils/utilities.py", line 77, in get_rays
    dirs_w = torch.sum(dirs * poses, axis=-1)
                       ~~~~~^~~~~~~
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

Originally posted by @a-lemus96 in #61

a-lemus96 commented 3 months ago

Previous test hardcoded tensor was not deleting, causing device mismatch...

> /home/lemus/projects/fs-nerf/src/utils/utilities.py(71)get_rays()
-> dirs = dirs/torch.norm(dirs, dim=-1, keepdim=True)
(Pdb) dirs.device
device(type='cuda', index=0)
(Pdb) n
> /home/lemus/projects/fs-nerf/src/utils/utilities.py(72)get_rays()
-> dirs = dirs[None, ..., None, :]
(Pdb) n
> /home/lemus/projects/fs-nerf/src/utils/utilities.py(74)get_rays()
-> poses = torch.rand(5, 4, 4)
(Pdb) n
> /home/lemus/projects/fs-nerf/src/utils/utilities.py(75)get_rays()
-> poses = poses.unsqueeze(-3) if poses.dim() == 2 else poses
(Pdb) n
> /home/lemus/projects/fs-nerf/src/utils/utilities.py(76)get_rays()
-> poses = poses[..., None, None, :3, :3]
(Pdb) n
> /home/lemus/projects/fs-nerf/src/utils/utilities.py(77)get_rays()
-> dirs_w = torch.sum(dirs * poses, axis=-1)
(Pdb) poses.device
device(type='cpu')
a-lemus96 commented 3 months ago

Poses are not being passed as tensors to get_rays function.

(Pdb) type(poses)
<class 'list'>
a-lemus96 commented 3 months ago

Poses are not being passed as tensors to get_rays function.

(Pdb) type(poses)
<class 'list'>

In fact, poses is a list containing two tensors: images and poses

(Pdb) poses[0].shape
torch.Size([50, 800, 800, 3])
(Pdb) poses[1].shape
torch.Size([50, 4, 4])
a-lemus96 commented 3 months ago

I removed device dependency in get_rays fn.