gsrelight / gs-relight

Official Code Release for SIGGRAPH Asia 2024 Paper: GS^3: Efficient Relighting with Triple Gaussian Splatting
GNU General Public License v3.0
85 stars 5 forks source link

Questions Regarding OLAT Dataset and Camera Model in 3DGS #3

Closed HarryPeverell closed 1 day ago

HarryPeverell commented 6 days ago

Hi!

Firstly, I would like to express my gratitude for your excellent work. I have a few questions regarding the OLAT dataset and the camera setup in 3DGS:

  1. OLAT Dataset Creation:
    Could you provide some insight into how the OLAT dataset was created? Specifically, I am curious about the requirements for the light source. Is there any particular characteristic that the light source should have (e.g., point light source or area light source)? If I already know the positions of the light source and camera for an image, how can I combine this information to generate the dataset?

  2. Camera Model in 3DGS:
    I noticed that 3DGS uses a pinhole camera model. If I want to use a different camera model, such as a fisheye camera or orthographic projection, aside from modifying the camera model in 3DGS, are there any additional changes required in the existing pipeline or workflow?

I understand that these might require some time to address, and I truly appreciate your help in answering these questions.

Thank you again for your time and support.

Best regards,
Harry

svbrdf commented 2 days ago

Thank you for your questions.

  1. Please refer to the following papers from us.

Learning Efficient Illumination Multiplexing for Joint Capture of Reflectance and Shape. Kaizhang Kang, Cihui Xie, Chengan He, Mingqi Yi, Minyi Gu, Zimin Chen, Kun Zhou and Hongzhi Wu. ACM Trans. on Graphics (Proc. SIGGRAPH Asia 2019), 38, 6 (Nov. 2019), 165.

Neural Reflectance Capture in the View-Illumination Domain. Kaizhang Kang, Minyi Gu, Cihui Xie, Xuanda Yang, Hongzhi Wu and Kun Zhou. IEEE Trans. on Visualization and Computer Graphics, 29, 2(Feb. 2023), 1450-1462.

It is possible to set up your own OLAT set with known point light position & power, and known camera parameters.

  1. Apply an SDK like OpenCV to transform your image to a new one, as if it is observed with a pin-hole camera.