search

PJLab-ADG / neuralsim

neuralsim: 3D surface reconstruction and simulation based on 3D neural rendering.
MIT License
582 stars 31 forks source link

NGP data format #13

Open xslittlegrass opened 11 months ago

xslittlegrass commented 11 months ago

I have some drive data in the Instant-NGP data format, what would be an easy way to try StreetSurf on that? Moreover, I have one front camera and two side cameras in my data, and the rolling shutter effect can be large for the side camera, does StreeSurf take account of the rolling shutter effect?

ventusff commented 11 months ago

Hi, by NGP data format do you mean the NeRF-format mentioned here ? We do have plans to integrate this type of data format.

Rolling shutter effects for LiDARs are partly implemented in Waymo Open Dataset. As for cameras, currently it's not implemented and require some efforts. Also, I think NGP / NeRF data format does not include information of rolling shutter effect. Can you provide some suggestions for standardize the data formats of rolling shutter effects and the ways to process them? Or I can start to do some research on it, but not recently.

xslittlegrass commented 11 months ago

Right, I mean NeRF-format in the link. That's great if you have plans to integrate that. It seems that NGP does have that rolling shutter in their code: https://github.com/NVlabs/instant-ngp/blob/7d5e858bba5885bbc593fc65e337e4410b992bef/src/nerf_loader.cu#L205