No module named 'depth_diff_gaussian_rasterization_min'

[ngc-shj]

Thank you for your great job. I installed the missing packages one by one[1], and solved the errors related to simple-kNN._C by defining the distCUDA2 function individually[2], and proceeded by trial and error, but I could not resolve the following error. Do you have any ideas on how to resolve this?

$ bash scripts/run_Pano2Room.sh 
/mnt/data/venv/pano2room/lib/python3.10/site-packages/kornia/feature/lightglue.py:44: FutureWarning: `torch.cuda.amp.custom_fwd(args...)` is deprecated. Please use `torch.amp.custom_fwd(args..., device_type='cuda')` instead.
  @torch.cuda.amp.custom_fwd(cast_inputs=torch.float32)
Detectron v2 is not installed
Traceback (most recent call last):
  File "/mnt/data/venv/pano2room/Pano2Room/pano2room.py", line 34, in <module>
    from gaussian_renderer import render
  File "/mnt/data/venv/pano2room/Pano2Room/gaussian_renderer/__init__.py", line 14, in <module>
    from depth_diff_gaussian_rasterization_min import GaussianRasterizationSettings, GaussianRasterizer
ModuleNotFoundError: No module named 'depth_diff_gaussian_rasterization_min'
$

[1] Build environment - venv & pip install(s)

python3 -m venv pano2room
cd $_
source bin/activate

git clone https://github.com/TrickyGo/Pano2Room
cd Pano2Room

pip install -r requirements.txt
#
pip install torch torchvision clip opencv-python open3d trimesh timm h5py kornia albumentations==0.5.2 webdataset omegaconf easydict pytorch-lightning diff-users icecream plyfile
pip install wheel
pip install git+https://github.com/facebookresearch/pytorch3d.git
pip install git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch
pip install git+https://github.com/JonathonLuiten/diff-gaussian-rasterization-w-depth

[2] git diff

--- a/scene/gaussian_model.py
+++ b/scene/gaussian_model.py
@@ -16,13 +16,19 @@ from plyfile import PlyData, PlyElement
 import torch
 from torch import nn

-from simple_knn._C import distCUDA2
+#from simple_knn._C import distCUDA2
 from utils.general import inverse_sigmoid, get_expon_lr_func, build_rotation
 from utils.system import mkdir_p
 from utils.sh import RGB2SH
 from utils.graphics import BasicPointCloud
 from utils.general import strip_symmetric, build_scaling_rotation

+def distCUDA2(points):
+    points_np = points.detach().cpu().float().numpy()
+    dists, inds = KDTree(points_np).query(points_np, k=4)
+    meanDists = (dists[:, 1:] ** 2).mean(1)
+
+    return torch.tensor(meanDists, dtype=points.dtype, device=points.device)

 class GaussianModel:
     def setup_functions(self):

[CameronFraser]

@ngc-shj

• For simple_knn use this repo
• Change import to diff_gaussian_rasterization
• Will also need transformers library
• Remove this loc
• Ensure additional packages are installed:
  • tifffile==2023.7.10
  • imageio==2.31.5
  • imageio-ffmpeg==0.4.7

[ngc-shj]

@CameronFraser Thank you for your answer. I followed your suggestions and tried again. When I executed the script, an error occurred.

Reading Training Transforms
Loading Training Cameras
Loading Preset Cameras
Number of points at initialisation :  2241274
Traceback (most recent call last):
  File "/mnt/data/venv/pano2room/Pano2Room/pano2room.py", line 606, in <module>
    pipeline.run()
  File "/mnt/data/venv/pano2room/Pano2Room/pano2room.py", line 566, in run
    self.train_GS()
  File "/mnt/data/venv/pano2room/Pano2Room/pano2room.py", line 429, in train_GS
    render_pkg = render(viewpoint_cam, self.gaussians, self.opt, self.background)
  File "/mnt/data/venv/pano2room/Pano2Room/gaussian_renderer/__init__.py", line 36, in render
    raster_settings = GaussianRasterizationSettings(
TypeError: GaussianRasterizationSettings.__new__() missing 1 required positional argument: 'debug'

[ngc-shj]

Remove this loc

I made the following corrections,

debug=False

and executed it, and it finished successfully!

100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2000/2000 [00:20<00:00, 96.28it/s]
Reading Training Transforms
Loading Training Cameras
Loading Preset Cameras
Number of points at initialisation :  2242039
Result saved at:  output/Pano2Room-results

[CameronFraser]

interesting, with what I have locally it didn't like the debug parameter :shrug: glad to hear it worked