JonasSchult / Mask3D

Mask3D predicts accurate 3D semantic instances achieving state-of-the-art on ScanNet, ScanNet200, S3DIS and STPLS3D.
MIT License
557 stars 111 forks source link

hydra.errors.HydraException: Error calling 'datasets.semseg.SemanticSegmentationDataset' : not available number labels, select from: 200, 200 #103

Open bh-cai opened 1 year ago

bh-cai commented 1 year ago

when I run the flow command, I got the issue: hydra.errors.HydraException: Error calling 'datasets.semseg.SemanticSegmentationDataset' : not available number labels, select from: 200, 200

main_instance_segmentation.py general.experiment_name=test1_scannet_val_query_150_topk_500_dbscan_0.95 general.project_name=scannet_eval general.checkpoint='checkpoints/scannet/scannet_val.ckpt' general.train_mode=false general.eval_on_segments=true general.train_on_segments=true model.num_queries=150 general.topk_per_image=500 general.use_dbscan=true general.dbscan_eps=0.95

/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/utilities/seed.py:55: UserWarning: No seed found, seed set to 2801433411 rank_zero_warn(f"No seed found, seed set to {seed}") Global seed set to 2801433411 EXPERIMENT ALREADY EXIST {'target': 'pytorch_lightning.loggers.WandbLogger', 'project': '${general.project_name}', 'name': '${general.experiment_name}', 'save_dir': '${general.save_dir}', 'entity': 'manjusaka_labs', 'resume': 'allow', 'id': '${general.experiment_name}'} wandb: Currently logged in as: bh_c (manjusaka_labs). Use wandb login --relogin to force relogin wandb: wandb version 0.15.4 is available! To upgrade, please run: wandb: $ pip install wandb --upgrade wandb: Tracking run with wandb version 0.15.0 wandb: Run data is saved locally in saved/test/test1_scannet_val_query_150_topk_500_dbscan_0.95/wandb/run-20230609_012749-test1_scannet_val_query_150_topk_500_dbscan_0.95 wandb: Run wandb offline to turn off syncing. wandb: Resuming run test1_scannet_val_query_150_topk_500_dbscan_0.95 wandb: ⭐️ View project at https://wandb.ai/manjusaka_labs/scannet_eval wandb: πŸš€ View run at https://wandb.ai/manjusaka_labs/scannet_eval/runs/test1_scannet_val_query_150_topk_500_dbscan_0.95 2023-06-09 01:27:54.018 | WARNING | utils.utils:load_checkpoint_with_missing_or_exsessive_keys:91 - Key not found, it will be initialized randomly: model.scene_min 2023-06-09 01:27:54.019 | WARNING | utils.utils:load_checkpoint_with_missing_or_exsessive_keys:91 - Key not found, it will be initialized randomly: model.scene_max 2023-06-09 01:27:54.145 | WARNING | utils.utils:load_checkpoint_with_missing_or_exsessive_keys:100 - criterion.empty_weight not in loaded checkpoint 2023-06-09 01:27:54.149 | WARNING | utils.utils:load_checkpoint_with_missing_or_exsessive_keys:115 - excessive key: model.scene_min 2023-06-09 01:27:54.149 | WARNING | utils.utils:load_checkpoint_with_missing_or_exsessive_keys:115 - excessive key: model.scene_max [2023-06-09 01:27:54,238][main][INFO] - {'general_train_mode': False, 'general_task': 'instance_segmentation', 'general_seed': None, 'general_checkpoint': 'checkpoints/scannet/scannet_val.ckpt', 'general_backbone_checkpoint': None, 'general_freeze_backbone': False, 'general_linear_probing_backbone': False, 'general_train_on_segments': True, 'general_eval_on_segments': True, 'general_filter_out_instances': False, 'general_save_visualizations': False, 'general_visualization_point_size': 20, 'general_decoder_id': -1, 'general_export': False, 'general_use_dbscan': True, 'general_ignore_class_threshold': 100, 'general_project_name': 'scannet_eval', 'general_workspace': 'jonasschult', 'general_experiment_name': 'test1_scannet_val_query_150_topk_500_dbscan_0.95', 'general_num_targets': 19, 'general_add_instance': True, 'general_dbscan_eps': 0.95, 'general_dbscan_min_points': 1, 'general_export_threshold': 0.0001, 'general_reps_per_epoch': 1, 'general_on_crops': False, 'general_scores_threshold': 0.0, 'general_iou_threshold': 1.0, 'general_area': 5, 'general_eval_inner_core': -1, 'general_topk_per_image': 500, 'general_ignore_mask_idx': [], 'general_max_batch_size': 99999999, 'general_save_dir': 'saved/test/test1_scannet_val_query_150_topk_500_dbscan_0.95', 'general_gpus': 1, 'data_train_mode': 'train', 'data_validation_mode': 'validation', 'data_test_mode': 'validation', 'data_ignore_label': 255, 'data_add_raw_coordinates': True, 'data_add_colors': True, 'data_add_normals': False, 'data_in_channels': 3, 'data_num_labels': 20, 'data_add_instance': True, 'data_task': 'instance_segmentation', 'data_pin_memory': False, 'data_num_workers': 4, 'data_batch_size': 5, 'data_test_batch_size': 1, 'data_cache_data': False, 'data_voxel_size': 0.02, 'data_reps_per_epoch': 1, 'data_cropping': False, 'data_cropping_args_min_points': 30000, 'data_cropping_args_aspect': 0.8, 'data_cropping_args_min_crop': 0.5, 'data_cropping_args_max_crop': 1.0, 'data_crop_min_size': 20000, 'data_crop_length': 6.0, 'data_cropping_v1': True, 'data_train_dataloadertarget_': 'torch.utils.data.DataLoader', 'data_train_dataloader_shuffle': True, 'data_train_dataloader_pin_memory': False, 'data_train_dataloader_num_workers': 4, 'data_train_dataloader_batch_size': 5, 'data_validation_dataloadertarget_': 'torch.utils.data.DataLoader', 'data_validation_dataloader_shuffle': False, 'data_validation_dataloader_pin_memory': False, 'data_validation_dataloader_num_workers': 4, 'data_validation_dataloader_batch_size': 1, 'data_test_dataloadertarget_': 'torch.utils.data.DataLoader', 'data_test_dataloader_shuffle': False, 'data_test_dataloader_pin_memory': False, 'data_test_dataloader_num_workers': 4, 'data_test_dataloader_batch_size': 1, 'data_train_datasettarget_': 'datasets.semseg.SemanticSegmentationDataset', 'data_train_dataset_dataset_name': 'scannet', 'data_train_dataset_data_dir': 'data/processed/scannet', 'data_train_dataset_image_augmentations_path': 'conf/augmentation/albumentations_aug.yaml', 'data_train_dataset_volume_augmentations_path': 'conf/augmentation/volumentations_aug.yaml', 'data_train_dataset_label_db_filepath': 'data/processed/scannet/label_database.yaml', 'data_train_dataset_color_mean_std': 'data/processed/scannet/color_mean_std.yaml', 'data_train_dataset_data_percent': 1.0, 'data_train_dataset_mode': 'train', 'data_train_dataset_ignore_label': 255, 'data_train_dataset_num_labels': 20, 'data_train_dataset_add_raw_coordinates': True, 'data_train_dataset_add_colors': True, 'data_train_dataset_add_normals': False, 'data_train_dataset_add_instance': True, 'data_train_dataset_instance_oversampling': 0.0, 'data_train_dataset_place_around_existing': False, 'data_train_dataset_point_per_cut': 0, 'data_train_dataset_max_cut_region': 0, 'data_train_dataset_flip_in_center': False, 'data_train_dataset_noise_rate': 0, 'data_train_dataset_resample_points': 0, 'data_train_dataset_add_unlabeled_pc': False, 'data_train_dataset_cropping': False, 'data_train_dataset_cropping_args_min_points': 30000, 'data_train_dataset_cropping_args_aspect': 0.8, 'data_train_dataset_cropping_args_min_crop': 0.5, 'data_train_dataset_cropping_args_max_crop': 1.0, 'data_train_dataset_is_tta': False, 'data_train_dataset_crop_min_size': 20000, 'data_train_dataset_crop_length': 6.0, 'data_train_dataset_filter_out_classes': [0, 1], 'data_train_dataset_label_offset': 2, 'data_validation_datasettarget_': 'datasets.semseg.SemanticSegmentationDataset', 'data_validation_dataset_dataset_name': 'scannet', 'data_validation_dataset_data_dir': 'data/processed/scannet', 'data_validation_dataset_image_augmentations_path': None, 'data_validation_dataset_volume_augmentations_path': None, 'data_validation_dataset_label_db_filepath': 'data/processed/scannet/label_database.yaml', 'data_validation_dataset_color_mean_std': 'data/processed/scannet/color_mean_std.yaml', 'data_validation_dataset_data_percent': 1.0, 'data_validation_dataset_mode': 'validation', 'data_validation_dataset_ignore_label': 255, 'data_validation_dataset_num_labels': 20, 'data_validation_dataset_add_raw_coordinates': True, 'data_validation_dataset_add_colors': True, 'data_validation_dataset_add_normals': False, 'data_validation_dataset_add_instance': True, 'data_validation_dataset_cropping': False, 'data_validation_dataset_is_tta': False, 'data_validation_dataset_crop_min_size': 20000, 'data_validation_dataset_crop_length': 6.0, 'data_validation_dataset_filter_out_classes': [0, 1], 'data_validation_dataset_label_offset': 2, 'data_test_dataset_target': 'datasets.semseg.SemanticSegmentationDataset', 'data_test_dataset_dataset_name': 'scannet', 'data_test_dataset_data_dir': 'data/processed/scannet', 'data_test_dataset_image_augmentations_path': None, 'data_test_dataset_volume_augmentations_path': None, 'data_test_dataset_label_db_filepath': 'data/processed/scannet/label_database.yaml', 'data_test_dataset_color_mean_std': 'data/processed/scannet/color_mean_std.yaml', 'data_test_dataset_data_percent': 1.0, 'data_test_dataset_mode': 'validation', 'data_test_dataset_ignore_label': 255, 'data_test_dataset_num_labels': 20, 'data_test_dataset_add_raw_coordinates': True, 'data_test_dataset_add_colors': True, 'data_test_dataset_add_normals': False, 'data_test_dataset_add_instance': True, 'data_test_dataset_cropping': False, 'data_test_dataset_is_tta': False, 'data_test_dataset_crop_min_size': 20000, 'data_test_dataset_crop_length': 6.0, 'data_test_dataset_filter_out_classes': [0, 1], 'data_test_dataset_label_offset': 2, 'data_train_collationtarget_': 'datasets.utils.VoxelizeCollate', 'data_train_collation_ignore_label': 255, 'data_train_collation_voxel_size': 0.02, 'data_train_collation_mode': 'train', 'data_train_collation_small_crops': False, 'data_train_collation_very_small_crops': False, 'data_train_collation_batch_instance': False, 'data_train_collation_probing': False, 'data_train_collation_task': 'instance_segmentation', 'data_train_collation_ignore_class_threshold': 100, 'data_train_collation_filter_out_classes': [0, 1], 'data_train_collation_label_offset': 2, 'data_train_collation_num_queries': 150, 'data_validation_collationtarget_': 'datasets.utils.VoxelizeCollate', 'data_validation_collation_ignore_label': 255, 'data_validation_collation_voxel_size': 0.02, 'data_validation_collation_mode': 'validation', 'data_validation_collation_batch_instance': False, 'data_validation_collation_probing': False, 'data_validation_collation_task': 'instance_segmentation', 'data_validation_collation_ignore_class_threshold': 100, 'data_validation_collation_filter_out_classes': [0, 1], 'data_validation_collation_label_offset': 2, 'data_validation_collation_num_queries': 150, 'data_test_collation_target': 'datasets.utils.VoxelizeCollate', 'data_test_collation_ignore_label': 255, 'data_test_collation_voxel_size': 0.02, 'data_test_collation_mode': 'validation', 'data_test_collation_batch_instance': False, 'data_test_collation_probing': False, 'data_test_collation_task': 'instance_segmentation', 'data_test_collation_ignore_class_threshold': 100, 'data_test_collation_filter_out_classes': [0, 1], 'data_test_collation_label_offset': 2, 'data_test_collation_num_queries': 150, 'logging': [{'target': 'pytorch_lightning.loggers.WandbLogger', 'project': 'scannet_eval', 'name': 'test1_scannet_val_query_150_topk_500_dbscan_0.95', 'save_dir': 'saved/test/test1_scannet_val_query_150_topk_500_dbscan_0.95', 'entity': 'manjusaka_labs', 'resume': 'allow', 'id': 'test1_scannet_val_query_150_topk_500_dbscan_0.95'}], 'modeltarget_': 'models.Mask3D', 'model_hidden_dim': 128, 'model_dim_feedforward': 1024, 'model_num_queries': 150, 'model_num_heads': 8, 'model_num_decoders': 3, 'model_dropout': 0.0, 'model_pre_norm': False, 'model_use_level_embed': False, 'model_normalize_pos_enc': True, 'model_positional_encoding_type': 'fourier', 'model_gauss_scale': 1.0, 'model_hlevels': [0, 1, 2, 3], 'model_non_parametric_queries': True, 'model_random_query_both': False, 'model_random_normal': False, 'model_random_queries': False, 'model_use_np_features': False, 'model_sample_sizes': [200, 800, 3200, 12800, 51200], 'model_max_sample_size': False, 'model_shared_decoder': True, 'model_num_classes': 19, 'model_train_on_segments': True, 'model_scatter_type': 'mean', 'model_voxel_size': 0.02, 'model_config_backbonetarget_': 'models.Res16UNet34C', 'model_config_backbone_config_dialations': [1, 1, 1, 1], 'model_config_backbone_config_conv1_kernel_size': 5, 'model_config_backbone_config_bn_momentum': 0.02, 'model_config_backbone_in_channels': 3, 'model_config_backbone_out_channels': 20, 'model_config_backbone_out_fpn': True, 'metricstarget_': 'models.metrics.ConfusionMatrix', 'metrics_num_classes': 20, 'metrics_ignore_label': 255, 'optimizertarget_': 'torch.optim.AdamW', 'optimizer_lr': 0.0001, 'scheduler_scheduler_target': 'torch.optim.lr_scheduler.OneCycleLR', 'scheduler_scheduler_max_lr': 0.0001, 'scheduler_scheduler_epochs': 601, 'scheduler_scheduler_steps_per_epoch': -1, 'scheduler_pytorch_lightning_params_interval': 'step', 'trainer_deterministic': False, 'trainer_max_epochs': 601, 'trainer_min_epochs': 1, 'trainer_resume_from_checkpoint': 'saved/test/test1_scannet_val_query_150_topk_500_dbscan_0.95/last-epoch.ckpt', 'trainer_check_val_every_n_epoch': 50, 'trainer_num_sanity_val_steps': 2, 'callbacks': [{'target': 'pytorch_lightning.callbacks.ModelCheckpoint', 'monitor': 'val_mean_ap_50', 'save_last': True, 'save_top_k': 1, 'mode': 'max', 'dirpath': 'saved/test/test1_scannet_val_query_150_topk_500_dbscan_0.95', 'filename': '{epoch}-{val_mean_ap_50:.3f}', 'every_n_epochs': 1}, {'target': 'pytorch_lightning.callbacks.LearningRateMonitor'}], 'matchertarget_': 'models.matcher.HungarianMatcher', 'matcher_cost_class': 2.0, 'matcher_cost_mask': 5.0, 'matcher_cost_dice': 2.0, 'matcher_num_points': -1, 'loss_target': 'models.criterion.SetCriterion', 'loss_num_classes': 19, 'loss_eos_coef': 0.1, 'loss_losses': ['labels', 'masks'], 'loss_num_points': -1, 'loss_oversample_ratio': 3.0, 'loss_importance_sample_ratio': 0.75, 'loss_class_weights': -1} /root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/accelerator_connector.py:446: LightningDeprecationWarning: Setting Trainer(gpus=1) is deprecated in v1.7 and will be removed in v2.0. Please use Trainer(accelerator='gpu', devices=1) instead. rank_zero_deprecation( /root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/checkpoint_connector.py:52: LightningDeprecationWarning: Setting Trainer(resume_from_checkpoint=) is deprecated in v1.5 and will be removed in v1.7. Please pass Trainer.fit(ckpt_path=) directly instead. rank_zero_deprecation( /root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/callback_connector.py:57: LightningDeprecationWarning: Setting Trainer(weights_save_path=) has been deprecated in v1.6 and will be removed in v1.8. Please pass dirpath directly to the ModelCheckpoint callback rank_zero_deprecation( GPU available: True (cuda), used: True TPU available: False, using: 0 TPU cores IPU available: False, using: 0 IPUs HPU available: False, using: 0 HPUs /home/mylabs/Mask3D/datasets/semseg.py:696: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details. file = yaml.load(f) Traceback (most recent call last): File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/utils.py", line 63, in call return _instantiate_class(type_or_callable, config, *args, *kwargs) File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 500, in _instantiate_class return clazz(args, **final_kwargs) File "/home/mylabs/Mask3D/datasets/semseg.py", line 218, in init self._labels = self._select_correct_labels(labels, num_labels) File "/home/mylabs/Mask3D/datasets/semseg.py", line 724, in _select_correct_labels raise ValueError(msg) ValueError: not available number labels, select from: 200, 200

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 198, in run_and_report return func() File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 347, in lambda: hydra.run( File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 107, in run return run_job( File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/core/utils.py", line 128, in run_job ret.return_value = task_function(task_cfg) File "/home/mylabs/Mask3D/main_instance_segmentation.py", line 110, in main test(cfg) File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/main.py", line 27, in decorated_main return task_function(cfg_passthrough) File "/home/mylabs/Mask3D/main_instance_segmentation.py", line 100, in test runner.test(model) File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 864, in test return self._call_and_handle_interrupt(self._test_impl, model, dataloaders, ckpt_path, verbose, datamodule) File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 650, in _call_and_handle_interrupt return trainer_fn(*args, *kwargs) File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 911, in _test_impl results = self._run(model, ckpt_path=self.ckpt_path) File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1097, in _run self._data_connector.prepare_data() File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/data_connector.py", line 120, in prepare_data self.trainer._call_lightning_module_hook("prepare_data") File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1552, in _call_lightning_module_hook output = fn(args, **kwargs) File "/home/mylabs/Mask3D/trainer/trainer.py", line 1269, in prepare_data self.train_dataset = hydra.utils.instantiate( File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/utils.py", line 70, in call raise HydraException(f"Error calling '{cls}' : {e}") from e hydra.errors.HydraException: Error calling 'datasets.semseg.SemanticSegmentationDataset' : not available number labels, select from: 200, 200

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/mylabs/Mask3D/main_instance_segmentation.py", line 114, in main() File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/main.py", line 32, in decorated_main _run_hydra( File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 346, in _run_hydra run_and_report( File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 267, in run_and_report print_exception(etype=None, value=ex, tb=final_tb) # type: ignore TypeError: print_exception() got an unexpected keyword argument 'etype'

Can you tell me how to solve the problem, thank you very much!

xibi777 commented 1 year ago

I encountered the same question! Did you solve it?

JonasSchult commented 1 year ago

Hi!

Is it possible that your generated dataset is ScanNet200 and not ScanNet?

Best, Jonas

bh-cai commented 1 year ago

Your reply and help are greatly appreciated! However, I regenrated the scannet data by " python python -m datasets.preprocessing.scannet_preprocessing preprocess \ --data_dir="./data/raw/scannet/scannet" \ --save_dir="./data/processed/scannet" \ --git_repo="./data/raw/scannet/ScanNet" \ --scannet200=False" the result of data structure like flow: ScanNet200 β”œβ”€β”€ instance_gt / β”‚ β”œβ”€β”€ train / β”‚ β”‚ β”œβ”€β”€ scene0000_00.txt
β”‚ β”‚ β”œβ”€β”€ ... β”‚ β”‚ └── scene0706_00.txt
β”‚ └── validation /
β”‚ β”œβ”€β”€ scene0011_00.txt
β”‚ β”œβ”€β”€ ... β”‚ └── scene0704_01.txt β”œβ”€β”€ train /
β”‚ β”œβ”€β”€ 0000_00.npy
β”‚ β”œβ”€β”€ ... β”‚ └── 0706_00.npy β”œβ”€β”€ validation /
β”‚ β”œβ”€β”€ 0011_00.npy
β”‚ β”œβ”€β”€ ... β”‚ └── 0704_01.npy β”œβ”€β”€ test /
β”‚ β”œβ”€β”€ 0707_00.npy
β”‚ β”œβ”€β”€ ... β”‚ └── 0806_00.npy
β”œβ”€β”€ color_mean_std.yaml 137B β”œβ”€β”€ label_database.yaml 15KB β”œβ”€β”€ test_database.yaml 24KB β”œβ”€β”€ train_database.yaml 1.19M β”œβ”€β”€ validation_database.yaml 319KB └── train_validation_database.yaml 1.50M

and regenerated the scannet200 data by

"python -m datasets.preprocessing.scannet_preprocessing preprocess \ --data_dir="./data/raw/scannet/scannet" \ --save_dir="./data/processed/scannet200" \ --git_repo="./data/raw/scannet/ScanNet" \ --scannet200=true" the result of data structure like flow: ScanNet200 β”œβ”€β”€ instance_gt / β”‚ β”œβ”€β”€ train / β”‚ β”‚ β”œβ”€β”€ scene0000_00.txt
β”‚ β”‚ β”œβ”€β”€ ... β”‚ β”‚ └── scene0706_00.txt
β”‚ └── validation /
β”‚ β”œβ”€β”€ scene0011_00.txt
β”‚ β”œβ”€β”€ ... β”‚ └── scene0704_01.txt β”œβ”€β”€ train /
β”‚ β”œβ”€β”€ 0000_00.npy
β”‚ β”œβ”€β”€ ... β”‚ └── 0706_00.npy β”œβ”€β”€ validation /
β”‚ β”œβ”€β”€ 0011_00.npy
β”‚ β”œβ”€β”€ ... β”‚ └── 0704_01.npy β”œβ”€β”€ test /
β”‚ β”œβ”€β”€ 0707_00.npy
β”‚ β”œβ”€β”€ ... β”‚ └── 0806_00.npy
β”œβ”€β”€ color_mean_std.yaml 137B β”œβ”€β”€ label_database.yaml 15KB β”œβ”€β”€ test_database.yaml 23KB β”œβ”€β”€ train_database.yaml 1.17M β”œβ”€β”€ validation_database.yaml 319KB └── train_validation_database.yaml 1.14M I used the same raw dataset "--data_dir="./data/raw/scannet/scannet" ",and just make the different command like " --scannet200=False" or " --scannet200=true". In the end, I got the same structure output data, and there just have the different on the size of data. So, if I used a wrong command? I not very understand clearly. Looking forward to your reply and help,best wish for you!

Hi!

Is it possible that your generated dataset is ScanNet200 and not ScanNet?

Best, Jonas

bh-cai commented 1 year ago

And when I run the train command as "python main_instance_segmentation.py" or "python main_instance_segmentation.py \ general.checkpoint='/home/mylabs/Mask3D/checkpoints/s3dis/from_scratch/area1.ckpt' general.train_mode=false" there was the same error.

bh-cai commented 1 year ago

I encountered the same question! Did you solve it?

not yet.

Haiyan-Chris-Wang commented 1 year ago

I have the similar problem and hope the author could help to explain it. Many thanks! @JonasSchult

Mask3D/main_instance_segmentation.py", line 79, in train runner = Trainer( File "/usr2/.local/lib/python3.10/site-packages/pytorch_lightning/utilities/argparse.py", line 345, in insert_env_defaults return fn(self, kwargs) File "/usr2/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 485, in init self._callback_connector.on_trainer_init( File "/usr2/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/callback_connector.py", line 89, in on_trainer_init self.trainer.callbacks.extend(_configure_external_callbacks()) File "/usr2/.local/lib/python3.10/site-packages/pytorch_lightning/trainer/connectors/callback_connector.py", line 265, in _configure_external_callbacks factories = entry_points(group=group) # type: ignore[call-arg] File "/opt/conda/envs/mask3d/lib/python3.10/importlib/metadata/init.py", line 1021, in entry_points return SelectableGroups.load(eps).select(params) File "/opt/conda/envs/mask3d/lib/python3.10/importlib/metadata/init.py", line 459, in load ordered = sorted(eps, key=by_group) File "/opt/conda/envs/mask3d/lib/python3.10/importlib/metadata/init.py", line 1018, in eps = itertools.chain.from_iterable( File "/opt/conda/envs/mask3d/lib/python3.10/importlib/metadata/_itertools.py", line 16, in unique_everseen k = key(element) File "/opt/conda/envs/mask3d/lib/python3.10/importlib/metadata/init.py", line 943, in _normalized_name or super()._normalized_name File "/opt/conda/envs/mask3d/lib/python3.10/importlib/metadata/init.py", line 622, in _normalizedname return Prepared.normalize(self.name) File "/opt/conda/envs/mask3d/lib/python3.10/importlib/metadata/init.py", line 871, in normalize return re.sub(r"[-.]+", "-", name).lower().replace('-', '_') File "/opt/conda/envs/mask3d/lib/python3.10/re.py", line 209, in sub return _compile(pattern, flags).sub(repl, string, count) TypeError: expected string or bytes-like object

During handling of the above exception, another exception occurred:

Mask3D/main_instance_segmentation.py", line 116, in main()
File "/opt/conda/envs/mask3d/lib/python3.10/site-packages/hydra/main.py", line 32, in decorated_main _run_hydra( File "/opt/conda/envs/mask3d/lib/python3.10/site-packages/hydra/_internal/utils.py", line 346, in _run_hydra run_and_report(
File "/opt/conda/envs/mask3d/lib/python3.10/site-packages/hydra/_internal/utils.py", line 267, in run_and_report print_exception(etype=None, value=ex, tb=final_tb) # type: ignore
TypeError: print_exception() got an unexpected keyword argument 'etype'

Jiayuan-Gu commented 1 year ago

It seems that the issue results from https://github.com/JonasSchult/Mask3D/blob/3db966df2c021c3361bd6eed56121428b3e7a21d/datasets/semseg.py#L699-L720. The label_database.yaml is the same for both ScanNet and ScanNet200 after preprocessing.

bh-cai commented 1 year ago

Thank you for your reply, but I don't understand how to change the code to solve this problem. May I get some advice from you?

Jiayuan-Gu commented 1 year ago

You just need to rerun scannet data preprocessing with --scannet200=False/True instead of --scannet200=false/true. See my pull request: #111

bh-cai commented 1 year ago

Thank you very much, I have solved the issue. However, I have got another problem with the flowing: "pytorch_lightning.utilities.exceptions.MisconfigurationException: ModelCheckpoint(monitor='val_mean_ap_50') could not find the monitored key in the returned metrics: ['train_loss_ce', 'train_loss_mask', 'train_loss_dice', 'train_loss_ce_0', 'train_loss_mask_0', 'train_loss_dice_0', 'train_loss_ce_1', 'train_loss_mask_1', 'train_loss_dice_1', 'train_loss_ce_2', 'train_loss_mask_2', 'train_loss_dice_2', 'train_loss_ce_3', 'train_loss_mask_3', 'train_loss_dice_3', 'train_loss_ce_4', 'train_loss_mask_4', 'train_loss_dice_4', 'train_loss_ce_5', 'train_loss_mask_5', 'train_loss_dice_5', 'train_loss_ce_6', 'train_loss_mask_6', 'train_loss_dice_6', 'train_loss_ce_7', 'train_loss_mask_7', 'train_loss_dice_7', 'train_loss_ce_8', 'train_loss_mask_8', 'train_loss_dice_8', 'train_loss_ce_9', 'train_loss_mask_9', 'train_loss_dice_9', 'train_loss_ce_10', 'train_loss_mask_10', 'train_loss_dice_10', 'train_loss_ce_11', 'train_loss_mask_11', 'train_loss_dice_11', 'train_mean_loss_ce', 'train_mean_loss_mask', 'train_mean_loss_dice', 'epoch', 'step']. HINT: Did you call log('val_mean_ap_50', value) in the LightningModule? "

Epoch 49: 100%|β–ˆβ–ˆβ–ˆβ–ˆ| 1513/1513 [1:56:39<00:00, 4.63s/it, loss=43.9, v_num=TION] Traceback (most recent call last): File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 198, in run_and_report return func() File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 347, in lambda: hydra.run( File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 107, in run return run_job( File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/core/utils.py", line 128, in run_job ret.return_value = task_function(task_cfg) File "/home/mylabs/Mask3D/main_instance_segmentation.py", line 108, in main train(cfg) File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/main.py", line 27, in decorated_main return task_function(cfg_passthrough) File "/home/mylabs/Mask3D/main_instance_segmentation.py", line 84, in train runner.fit(model) File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 696, in fit self._call_and_handle_interrupt( File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 650, in _call_and_handle_interrupt return trainer_fn(*args, kwargs) File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 737, in _fit_impl results = self._run(model, ckpt_path=self.ckpt_path) File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1168, in _run results = self._run_stage() File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1254, in _run_stage return self._run_train() File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1285, in _run_train self.fit_loop.run() File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/loops/loop.py", line 200, in run self.advance(*args, *kwargs) File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/loops/fit_loop.py", line 270, in advance self._outputs = self.epoch_loop.run(self._data_fetcher) File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/loops/loop.py", line 201, in run self.on_advance_end() File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/loops/epoch/training_epoch_loop.py", line 241, in on_advance_end self._run_validation() File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/loops/epoch/training_epoch_loop.py", line 299, in _run_validation self.val_loop.run() File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/loops/loop.py", line 207, in run output = self.on_run_end() File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 201, in on_run_end self._on_evaluation_end() File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/loops/dataloader/evaluation_loop.py", line 265, in _on_evaluation_end self.trainer._call_callback_hooks(hook_name, args, kwargs) File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/trainer/trainer.py", line 1599, in _call_callback_hooks fn(self, self.lightning_module, *args, **kwargs) File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/callbacks/model_checkpoint.py", line 319, in on_validation_end self._save_topk_checkpoint(trainer, monitor_candidates) File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/pytorch_lightning/callbacks/model_checkpoint.py", line 380, in _save_topk_checkpoint raise MisconfigurationException(m) pytorch_lightning.utilities.exceptions.MisconfigurationException: ModelCheckpoint(monitor='val_mean_ap_50') could not find the monitored key in the returned metrics: ['train_loss_ce', 'train_loss_mask', 'train_loss_dice', 'train_loss_ce_0', 'train_loss_mask_0', 'train_loss_dice_0', 'train_loss_ce_1', 'train_loss_mask_1', 'train_loss_dice_1', 'train_loss_ce_2', 'train_loss_mask_2', 'train_loss_dice_2', 'train_loss_ce_3', 'train_loss_mask_3', 'train_loss_dice_3', 'train_loss_ce_4', 'train_loss_mask_4', 'train_loss_dice_4', 'train_loss_ce_5', 'train_loss_mask_5', 'train_loss_dice_5', 'train_loss_ce_6', 'train_loss_mask_6', 'train_loss_dice_6', 'train_loss_ce_7', 'train_loss_mask_7', 'train_loss_dice_7', 'train_loss_ce_8', 'train_loss_mask_8', 'train_loss_dice_8', 'train_loss_ce_9', 'train_loss_mask_9', 'train_loss_dice_9', 'train_loss_ce_10', 'train_loss_mask_10', 'train_loss_dice_10', 'train_loss_ce_11', 'train_loss_mask_11', 'train_loss_dice_11', 'train_mean_loss_ce', 'train_mean_loss_mask', 'train_mean_loss_dice', 'epoch', 'step']. HINT: Did you call log('val_mean_ap_50', value) in the LightningModule?

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/mylabs/Mask3D/main_instance_segmentation.py", line 114, in main() File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/main.py", line 32, in decorated_main _run_hydra( File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 346, in _run_hydra run_and_report( File "/root/anaconda3/envs/mask3d_cuda113/lib/python3.10/site-packages/hydra/_internal/utils.py", line 267, in run_and_report print_exception(etype=None, value=ex, tb=final_tb) # type: ignore TypeError: print_exception() got an unexpected keyword argument 'etype' wandb: Waiting for W&B process to finish... (failed 1). Press Control-C to abort syncing.

I am not very clear about the problem above, may I get some help from you, if so, it would my best greatest.

asadjan1801 commented 1 year ago

I have the same problem. Did you ever find solution for this problem.

zoeyliu1999 commented 11 months ago

Hey, I just found that the function traceback.print_exception() has been changed in python v3.10, where the 'etype' parameter has been renamed to 'exc' and is now positional-only.

So I rewrite the file "/Users/.../python3.10/site-packages/hydra/_internal/utils.py", line 267 from

print_exception(etype=None, value=ex, tb=final_tb)

to

print_exception(None, value=ex, tb=final_tb)

And things go well. The package's change log can be found in website).