Closed jelleopard closed 1 year ago
Carl-Carl
commented
1 year ago Hi, thank you for your feedback. Could you please provide me with more info about the error? Are you trying to simply reproduce our experiment result on the Kitti dataset or use our model on another dataset?
jelleopard
commented
1 year ago Thanks for the reply, I was reproduce the experiment on the kitti-0006 dataset. The following code block is where the error occurs：
_type2str = ["Car", None, "Truck"]
for class_id, type_id in enumerate(type_list):
type_id = int(type_id)
if type_id == -1 or output_obj[class_id] is None:
continue
mask = ray_class_id == class_id
index = id_z_vals_obj[intersection_map[mask, 0], intersection_map[mask, 1], :, :]
densities[index[..., 0], index[..., 1], 0] = output_obj[class_id]["field_outputs"][FieldHeadNames.DENSITY][
..., 0
]
rgbs[index[..., 0], index[..., 1], :] = output_obj[class_id]["field_outputs"][FieldHeadNames.RGB][..., :]
if self.use_semantic:
semantics[index[..., 0], index[..., 1], self.background_model.str2semantic[_type2str[type_id]]] = 1.0The error reported as follows：
File "/home/mars/nsg/models/scene_graph.py", line 519, in get_outputs
semantics[index[..., 0], index[..., 1], self.background_model.str2semantic[_type2str[type_id]]] = 1.0
IndexError: list index out of rangeA more specific error is this statement： The value of the type_id is out of bounds _type2str = ["Car", None, "Truck"] self.background_model.str2semantic[_type2str[type_id]]] = 1.0
AmazingRoad
commented
1 year ago @jelleopard Check your "type_id" or "_type2str[type_id]"? And check if the str2semantic is satisfied?
JiantengChen
commented
1 year ago Hi! Is this error occurring during the training process or before starting the training? If it's during the training process, can you provide more detailed error information?
jelleopard
commented
1 year ago Thanks for your reply, that's all information about of this error. this error occurring before starting the training
Hello author, the following problem arises when I use sematicNeRF, I need your help, thanks
File "/home/mars/nsg/models/scene_graph.py", line 519, in get_outputs semantics[index[..., 0], index[..., 1], self.background_model.str2semantic[_type2str[type_id]]] = 1.0 IndexError: list index out of range_type2str = ["Car", None, "Truck"] for class_id, type_id in enumerate(type_list): type_id = int(type_id) if type_id == -1 or output_obj[class_id] is None: continue mask = ray_class_id == class_id index = id_z_vals_obj[intersection_map[mask, 0], intersection_map[mask, 1], :, :] densities[index[..., 0], index[..., 1], 0] = output_obj[class_id]["field_outputs"][FieldHeadNames.DENSITY][ ..., 0 ] rgbs[index[..., 0], index[..., 1], :] = output_obj[class_id]["field_outputs"][FieldHeadNames.RGB][..., :] if self.use_semantic: semantics[index[..., 0], index[..., 1], self.background_model.str2semantic[_type2str[type_id]]] = 1.0KITTI_Recon_NSG_Car_Depth = MethodSpecification( config=TrainerConfig( method_name="nsg-kitti-car-depth-recon", steps_per_eval_image=STEPS_PER_EVAL_IMAGE, steps_per_eval_all_images=STEPS_PER_EVAL_ALL_IMAGES, steps_per_save=STEPS_PER_SAVE, max_num_iterations=MAX_NUM_ITERATIONS, save_only_latest_checkpoint=False, mixed_precision=False, use_grad_scaler=True, log_gradients=True, pipeline=NSGPipelineConfig( datamanager=NSGkittiDataManagerConfig( dataparser=NSGkittiDataParserConfig( use_car_latents=True, use_depth=True, # use_semantic=False, use_semantic=True, semantic_mask_classes=['Van', 'Undefined'], semantic_path=Path("/home/mars/data/kitti/panoptic_maps"), split_setting="reconstruction", car_object_latents_path=Path( "/home/mars/latents/KITTI-MOT/car-object-latents/latent_codes_car_van_truck.pt" ), car_nerf_state_dict_path=Path("/home/mars/latents/KITTI-MOT/car-nerf-state-dict/epoch_670.ckpt"), ), train_num_rays_per_batch=4096, eval_num_rays_per_batch=4096, camera_optimizer=CameraOptimizerConfig(mode="off"), ), model=SceneGraphModelConfig( background_model=SemanticNerfWModelConfig( num_proposal_iterations=1, num_proposal_samples_per_ray=[48], num_nerf_samples_per_ray=97, use_single_jitter=False, semantic_loss_weight=0.1 ), # background_model=NerfactoModelConfig(), mono_depth_loss_mult=0.05, depth_loss_mult=0, # use_sky_model=True, object_model_template=CarNeRFModelConfig(_target=CarNeRF), object_representation="class-wise", # object_representation="object-wise", object_ray_sample_strategy="remove-bg", ), ), optimizers={ "background_model": { "optimizer": RAdamOptimizerConfig(lr=1e-3, eps=1e-15), "scheduler": ExponentialDecaySchedulerConfig(lr_final=1e-5, max_steps=200000), }, "learnable_global": { "optimizer": RAdamOptimizerConfig(lr=1e-3, eps=1e-15), "scheduler": ExponentialDecaySchedulerConfig(lr_final=1e-5, max_steps=200000), }, "object_model": { "optimizer": RAdamOptimizerConfig(lr=5e-3, eps=1e-15), "scheduler": ExponentialDecaySchedulerConfig(lr_final=1e-5, max_steps=200000), }, # "sky_model": { # "optimizer": RAdamOptimizerConfig(lr=5e-3, eps=1e-15), # "scheduler": ExponentialDecaySchedulerConfig(lr_final=1e-5, max_steps=200000), # }, }, # viewer=ViewerConfig(num_rays_per_chunk=1 << 15), vis="wandb", # vis="tensorboard", ), description="Neural Scene Graph implementation with vanilla-NeRF model for backgruond and object models.", )
AmazingRoad
commented
1 year ago @jelleopard Could you share your reconstruction results with semantic maps? Thanks!
Hello author, the following problem arises when I use sematicNeRF, I need your help, thanks