some doubts about kitti data in training step

[xuyanging]

thank you for your work,

I am learning to reproduce kitti results, and I notice that you use car_object_latents named latent_codes_car_van_truck.pt, could you share this car_latents? I try to use car-object-latents/latent_codes06.pt to train, but effect is not so good. the car is not clear and all the vehicles are same.

Besides，another question is how to use the panoptic_maps you provided in training process, could you share your cacai_config about training with panoptic maps?

much thanks

[JiantengChen]

Hi! Thanks for your reply.

I am learning to reproduce kitti results, and I notice that you use car_object_latents named latent_codes_car_van_truck.pt, could you share this car_latents? I try to use car-object-latents/latent_codes06.pt to train, but effect is not so good. the car is not clear and all the vehicles are same.

You can download latent_codes_car_van_truck.pt here

Besides，another question is how to use the panoptic_maps you provided in training process, could you share your cacai_config about training with panoptic maps?

To use panoptic_maps, you should specify use_semantics and semantic_path. And you should use SemanticNerfWModelConfig as your background model. Below is an example config that uses panoptic maps.

CLICK ME

```python KITTI_Recon_NSG_Car_Depth = MethodSpecification( config=TrainerConfig( method_name="nsg-kitti-car-depth-recon", steps_per_eval_image=STEPS_PER_EVAL_IMAGE, steps_per_eval_all_images=STEPS_PER_EVAL_ALL_IMAGES, steps_per_save=STEPS_PER_SAVE, max_num_iterations=MAX_NUM_ITERATIONS, save_only_latest_checkpoint=False, mixed_precision=False, use_grad_scaler=True, log_gradients=True, pipeline=NSGPipelineConfig( datamanager=NSGkittiDataManagerConfig( dataparser=NSGkittiDataParserConfig( scale_factor=0.1, use_car_latents=True, use_depth=True, use_semantic=True, semantic_mask_classes=['Van', 'Undefined'], car_object_latents_path=Path( "/data1/chenjt/datasets/ckpts/pretrain/car_nerf/kitti_mot/latents/latent_codes06.pt" ), split_setting="reconstruction", car_nerf_state_dict_path=Path("/data1/chenjt/datasets/ckpts/pretrain/car_nerf/epoch_670.ckpt"), semantic_path=Path("/data/kitti-MOT/panoptic_maps") ), train_num_rays_per_batch=4096, eval_num_rays_per_batch=4096, camera_optimizer=CameraOptimizerConfig(mode="off"), ), model=SceneGraphModelConfig( background_model=SemanticNerfWModelConfig( num_proposal_iterations=1, num_proposal_samples_per_ray=[48], num_nerf_samples_per_ray=97, use_single_jitter=False, semantic_loss_weight=0.1 ), mono_depth_loss_mult=0.01, depth_loss_mult=0, use_sky_model=True, object_model_template=CarNeRFModelConfig(_target=CarNeRF), object_representation="class-wise", object_ray_sample_strategy="remove-bg", ), ), optimizers={ "background_model": { "optimizer": RAdamOptimizerConfig(lr=1e-3, eps=1e-15), "scheduler": ExponentialDecaySchedulerConfig(lr_final=1e-5, max_steps=200000), }, "sky_model": { "optimizer": RAdamOptimizerConfig(lr=3e-3, eps=1e-15), "scheduler": ExponentialDecaySchedulerConfig(lr_final=1e-5, max_steps=200000), }, "object_model": { "optimizer": RAdamOptimizerConfig(lr=5e-3, eps=1e-15), "scheduler": ExponentialDecaySchedulerConfig(lr_final=1e-5, max_steps=200000), }, }, # viewer=ViewerConfig(num_rays_per_chunk=1 << 15), vis="wandb", ), description="", ) ```

[AmazingRoad]

@xBeho1der hello, how to train a latent codes with my own dataset，considering that Car_studio is not open source. And will Car_studio release its code in the future?

[wuzirui]

The code will be released once it passes the review and is accepted.

@AmazingRoad https://github.com/lty2226262/Car_studio