Implementation status

[inuex35]

Working on spherical_render branch

Todo

• [x] Load Opensfm data for spherical camera
• [x] Render forward - Need to check pixels are in the right position
• [x] Render forward - Need to check if jacobian is okay
• [x] Render backward - Need to check mathematical correctness for backward

Current status Forward pass is not good

Sample dataset

https://dtbn.jp/6eEZeo8e (The url will expire in a week)

image

reconstruction

[inuex35]

Forward pass is look okay

Iterations 7000

Iterations 30000

rendering result is not good and need to check jacobian and backward

[inuex35]

Worked! I dont know why it's upside down.

https://github.com/user-attachments/assets/d9eff32d-0c4b-4c50-a2ed-080883eabe5e

Problem was the original dataset. It does not happen in another dataset.

[inuex35]

Real data is not as good as simulated data, but looks working well.

https://github.com/user-attachments/assets/192cc732-d8e6-4c3e-a9eb-45e89b315388

Rendering perspective from spherical model is okay but smoky because of large floaters under ground.

[weixiabing]

Excellent contribution, I tested the code on the OmniBlender dataset and found that the default strategy has a sharp rise in loss after the 3000th step and seems to reset the opacity every 100 steps after that, resulting in a poor encounter, whereas with the mcmc strategy my psnr was 5db higher than the results on the omnigs paper!

[inuex35]

Thank you for the validation! Indeed, some state-of-the-art functions like MCMC improve quality compared to the original implementation. I plan to try them on some datasets when I have time. If you've done any validation, please share it with everyone.

https://github.com/nerfstudio-project/gsplat/pull/457