PJLab-ADG / neuralsim

neuralsim: 3D surface reconstruction and simulation based on 3D neural rendering.
MIT License
582 stars 31 forks source link

[Question] Training steetsurf with several scenarios #36

Open jnskkmhr opened 10 months ago

jnskkmhr commented 10 months ago

Hi, thanks for releasing great work! I have a question regarding dataset config. I am currently using a custom dataset for streetsurf and it seems the model works fine on my dataset.

However, when I fuse scenarios where the ego vehicle drives a relatively similar location, the model performance drops compared to the model trained on a single scenario. (I cannot share the result since it is not supposed to be open dataset)

So, I wanted to ask if fusing multiple scenarios is supported in streetsurf. I just want to know if I am using your code in the correct way. If so, then I am guessing the performance drop is due to a localization error.

Below, I am attaching part of the config file as a reference. Looking forward to your reply. Thank you!

scenebank_cfg:
  scenarios:
    # fuse several scenarios into single big scenario
    - 602620834232f4002ce37c9f-602625556eece1001edabfd3-6026207ddbf4220033e68f78
    # use multiple scenarios
    # - 602620834232f4002ce37c9f
    # - 602625556eece1001edabfd3
    # - 6026207ddbf4220033e68f78
  observer_cfgs: 
    Camera:
      list: ${camera_list}
    RaysLidar:
      list: ${lidar_list}
  on_load:
    no_objects: true # Set to true to skip loading foreground objects into scene graph
    # joint_camlidar: true # !!! Convinient for NVS
    align_orientation: true
    # consider_distortion: true
    # joint_camlidar_equivalent_extr: true
    camera_front_name: camera_0