autonomousvision / mip-splatting

[CVPR'24 Best Student Paper] Mip-Splatting: Alias-free 3D Gaussian Splatting
https://niujinshuchong.github.io/mip-splatting/
Other
1.1k stars 71 forks source link

Training loss doesn't converge #60

Open nanasylum opened 6 days ago

nanasylum commented 6 days ago

Hi, thanks for ur work. My training loss does not converge. Initially I thought maybe its because i add some code in scene/dataset_readers.py/def getNerfppNorm(cam_info):

  def getNerfppNorm(cam_info):

      def get_center_and_diag(cam_centers):
          cam_centers = np.hstack(cam_centers)
          avg_cam_center = np.mean(cam_centers, axis=1, keepdims=True)
          center = avg_cam_center
          dist = np.linalg.norm(cam_centers - center, axis=0, keepdims=True)
          diagonal = np.max(dist)
          return center.flatten(), diagonal
      cam_centers = []
      for cam in cam_info:
          W2C = getWorld2View2(cam.R, cam.T)
          C2W = np.linalg.inv(W2C)
          cam_centers.append(C2W[:3, 3:4])
      center, diagonal = get_center_and_diag(cam_centers)
      radius = diagonal * 1.1

      # i add this sentence because https://github.com/graphdeco-inria/gaussian-splatting/issues/482
      ***radius = 100.0***

      translate = -center
      return {"translate": translate, "radius": radius} 

Then the loss is:

Number of points at initialisation :  194892 [19/11 09:53:28]
Computing 3D filter [19/11 09:53:28]
Training progress:   2%|█▋                                                                                   | 590/30000 [00:04<03:46, 129.59it/s, Loss=0.0824090]Computing 3D filter [19/11 09:53:33]
Training progress:   2%|█▉                                                                                   | 690/30000 [00:05<04:28, 109.05it/s, Loss=0.0827591]Computing 3D filter [19/11 09:53:34]
Training progress:   3%|██▏                                                                                  | 790/30000 [00:06<04:27, 109.26it/s, Loss=0.0760958]Computing 3D filter [19/11 09:53:35]
Training progress:   3%|██▌                                                                                  | 890/30000 [00:07<04:26, 109.39it/s, Loss=0.0704437]Computing 3D filter [19/11 09:53:36]
Training progress:   3%|██▊                                                                                  | 990/30000 [00:08<04:22, 110.63it/s, Loss=0.0701113]Computing 3D filter [19/11 09:53:36]
Training progress:   4%|███                                                                                 | 1090/30000 [00:09<04:24, 109.31it/s, Loss=0.0901082]Computing 3D filter [19/11 09:53:37]
Training progress:   4%|███▎                                                                                | 1190/30000 [00:10<04:24, 108.75it/s, Loss=0.0835246]Computing 3D filter [19/11 09:53:38]
Training progress:   4%|███▌                                                                                | 1290/30000 [00:11<04:14, 112.99it/s, Loss=0.0830248]Computing 3D filter [19/11 09:53:39]
Training progress:   5%|███▉                                                                                | 1390/30000 [00:12<04:19, 110.07it/s, Loss=0.0737786]Computing 3D filter [19/11 09:53:40]
Training progress:   5%|████▏                                                                               | 1490/30000 [00:13<04:03, 116.87it/s, Loss=0.0864849]Computing 3D filter [19/11 09:53:41]
Training progress:   5%|████▍                                                                               | 1590/30000 [00:14<04:16, 110.78it/s, Loss=0.0724047]Computing 3D filter [19/11 09:53:42]
Training progress:   6%|████▋                                                                               | 1690/30000 [00:15<04:10, 113.13it/s, Loss=0.0726402]Computing 3D filter [19/11 09:53:43]
Training progress:   6%|█████                                                                               | 1790/30000 [00:16<04:13, 111.44it/s, Loss=0.0727235]Computing 3D filter [19/11 09:53:44]
Training progress:   6%|█████▎                                                                              | 1890/30000 [00:16<04:05, 114.28it/s, Loss=0.0723223]Computing 3D filter [19/11 09:53:45]
Training progress:   7%|█████▌                                                                              | 1990/30000 [00:17<04:09, 112.18it/s, Loss=0.0916959]Computing 3D filter [19/11 09:53:46]
Training progress:   7%|█████▊                                                                              | 2090/30000 [00:18<04:07, 112.69it/s, Loss=0.0834687]Computing 3D filter [19/11 09:53:47]
Training progress:   7%|██████▏                                                                             | 2190/30000 [00:19<04:04, 113.96it/s, Loss=0.0808549]Computing 3D filter [19/11 09:53:48]
Training progress:   8%|██████▍                                                                             | 2290/30000 [00:20<04:18, 107.36it/s, Loss=0.0949283]Computing 3D filter [19/11 09:53:49]
Training progress:   8%|██████▋                                                                             | 2390/30000 [00:21<04:10, 110.33it/s, Loss=0.0753420]Computing 3D filter [19/11 09:53:49]
Training progress:   8%|██████▉                                                                             | 2490/30000 [00:22<04:03, 112.87it/s, Loss=0.0798656]Computing 3D filter [19/11 09:53:50]
Training progress:   9%|███████▎                                                                            | 2590/30000 [00:23<04:07, 110.80it/s, Loss=0.0655544]Computing 3D filter [19/11 09:53:51]
Training progress:   9%|███████▌                                                                            | 2690/30000 [00:24<04:13, 107.55it/s, Loss=0.0803324]Computing 3D filter [19/11 09:53:52]
Training progress:   9%|███████▊                                                                            | 2790/30000 [00:25<04:15, 106.57it/s, Loss=0.0741386]Computing 3D filter [19/11 09:53:53]
Training progress:  10%|████████                                                                            | 2890/30000 [00:26<04:10, 108.20it/s, Loss=0.0824648]Computing 3D filter [19/11 09:53:54]
Training progress:  10%|████████▎                                                                           | 2990/30000 [00:27<04:17, 104.93it/s, Loss=0.0754795]Computing 3D filter [19/11 09:53:55]
Training progress:  10%|████████▋                                                                           | 3090/30000 [00:28<04:21, 102.88it/s, Loss=0.0758464]Computing 3D filter [19/11 09:53:56]
Training progress:  11%|████████▉                                                                           | 3190/30000 [00:29<04:08, 107.72it/s, Loss=0.0709470]Computing 3D filter [19/11 09:53:57]
Training progress:  11%|█████████▏                                                                          | 3290/30000 [00:30<04:21, 102.22it/s, Loss=0.0776415]Computing 3D filter [19/11 09:53:58]
Training progress:  11%|█████████▍                                                                          | 3390/30000 [00:31<04:20, 102.32it/s, Loss=0.0744108]Computing 3D filter [19/11 09:53:59]
Training progress:  12%|█████████▊                                                                          | 3490/30000 [00:32<04:24, 100.18it/s, Loss=0.0787199]Computing 3D filter [19/11 09:54:00]
Training progress:  12%|██████████                                                                          | 3590/30000 [00:33<04:19, 101.86it/s, Loss=0.0707832]Computing 3D filter [19/11 09:54:01]
Training progress:  12%|██████████▎                                                                         | 3690/30000 [00:34<04:18, 101.88it/s, Loss=0.0671102]Computing 3D filter [19/11 09:54:02]
Training progress:  13%|██████████▌                                                                         | 3790/30000 [00:35<04:11, 104.35it/s, Loss=0.0731359]Computing 3D filter [19/11 09:54:03]
Training progress:  13%|██████████▉                                                                         | 3900/30000 [00:36<04:05, 106.42it/s, Loss=0.0803724]Computing 3D filter [19/11 09:54:04]
Training progress:  13%|███████████▏                                                                        | 4000/30000 [00:37<04:14, 102.25it/s, Loss=0.0684935]Computing 3D filter [19/11 09:54:05]
Training progress:  14%|███████████▌                                                                         | 4100/30000 [00:38<04:23, 98.45it/s, Loss=0.0917881]Computing 3D filter [19/11 09:54:06]
Training progress:  14%|███████████▉                                                                         | 4200/30000 [00:39<04:20, 99.13it/s, Loss=0.0809305]Computing 3D filter [19/11 09:54:07]
Training progress:  14%|████████████                                                                        | 4290/30000 [00:40<04:02, 106.21it/s, Loss=0.0665322]Computing 3D filter [19/11 09:54:08]
Training progress:  15%|████████████▎                                                                       | 4400/30000 [00:41<04:15, 100.03it/s, Loss=0.0726099]Computing 3D filter [19/11 09:54:10]
Training progress:  15%|████████████▌                                                                       | 4500/30000 [00:42<04:05, 103.68it/s, Loss=0.0734125]Computing 3D filter [19/11 09:54:11]
Training progress:  15%|████████████▉                                                                       | 4600/30000 [00:43<04:09, 101.92it/s, Loss=0.0737833]Computing 3D filter [19/11 09:54:12]
Training progress:  16%|█████████████▏                                                                      | 4690/30000 [00:44<04:12, 100.29it/s, Loss=0.0737864]Computing 3D filter [19/11 09:54:13]
Training progress:  16%|█████████████▍                                                                      | 4790/30000 [00:45<04:09, 101.17it/s, Loss=0.0764596]Computing 3D filter [19/11 09:54:14]
Training progress:  16%|█████████████▉                                                                       | 4900/30000 [00:46<04:15, 98.36it/s, Loss=0.0822290]Computing 3D filter [19/11 09:54:15]

Actually until final the loss remains in around 0.08, never change. So then i delete my added code. But the loss remains in around 0.08 Hope for your reply.