Closed lqlnbcr closed 1 year ago
input:
python run.py \
task.task_name=predict \
predict.src_las=tests/data/Sydney_dataset/newdata.las \
predict.output_dir=/tests/data/output_dir/ \
predict.gpus=0 \
datamodule.batch_size=2
return:
CONFIG
├── task
│ └── task_name: predict
│
├── seed
│ └── 12345
├── logger
│ └── comet:
│ _target_: pytorch_lightning.loggers.comet.CometLogger
│ api_key: ${oc.env:COMET_API_TOKEN}
│ workspace: ${oc.env:COMET_WORKSPACE}
│ project_name: ${oc.env:COMET_PROJECT_NAME}
│ experiment_name: '[V3.0.2-BS10xMAX40000pts] RandLaNet_base_run_FR_pyg_randla_net_NoRS'
│ auto_log_co2: false
│ disabled: false
│
├── trainer
│ └── _target_: pytorch_lightning.Trainer
│ gpus:
│ - 0
│ - 1
│ min_epochs: 100
│ max_epochs: 150
│ log_every_n_steps: 1
│ weights_summary: null
│ progress_bar_refresh_rate: 1
│ auto_lr_find: false
│ num_sanity_val_steps: 2
│ accumulate_grad_batches: 3
│ strategy: ddp
│ accelerator: gpu
│ num_processes: 2
│
├── model
│ └── optimizer:
│ _target_: functools.partial
│ _args_:
│ - ${get_method:torch.optim.Adam}
│ lr: ${model.lr}
│ lr_scheduler:
│ _target_: functools.partial
│ _args_:
│ - ${get_method:torch.optim.lr_scheduler.ReduceLROnPlateau}
│ mode: min
│ factor: 0.5
│ patience: 20
│ cooldown: 5
│ verbose: true
│ criterion:
│ _target_: torch.nn.CrossEntropyLoss
│ label_smoothing: 0.0
│ _target_: myria3d.models.model.Model
│ d_in: ${dataset_description.d_in}
│ num_classes: ${dataset_description.num_classes}
│ ckpt_path: null
│ neural_net_class_name: PyGRandLANet
│ neural_net_hparams:
│ num_features: ${model.d_in}
│ num_classes: ${model.num_classes}
│ num_neighbors: 16
│ decimation: 4
│ return_logits: true
│ interpolation_k: ${predict.interpolator.interpolation_k}
│ num_workers: 4
│ iou:
│ _target_: functools.partial
│ _args_:
│ - ${get_method:torchmetrics.JaccardIndex}
│ - ${model.num_classes}
│ absent_score: 1.0
│ momentum: 0.9
│ monitor: val/loss_epoch
│ lr: 0.003933709606504788
│
├── datamodule
│ └── transforms:
│ preparations:
│ train:
│ TargetTransform:
│ _target_: myria3d.pctl.transforms.transforms.TargetTransform
│ _args_:
│ - ${dataset_description.classification_preprocessing_dict}
│ - ${dataset_description.classification_dict}
│ DropPointsByClass:
│ _target_: myria3d.pctl.transforms.transforms.DropPointsByClass
│ GridSampling:
│ _target_: torch_geometric.transforms.GridSampling
│ _args_:
│ - 0.25
│ MinimumNumNodes:
│ _target_: myria3d.pctl.transforms.transforms.MinimumNumNodes
│ _args_:
│ - 300
│ MaximumNumNodes:
│ _target_: myria3d.pctl.transforms.transforms.MaximumNumNodes
│ _args_:
│ - 40000
│ Center:
│ _target_: torch_geometric.transforms.Center
│ eval:
│ TargetTransform:
│ _target_: myria3d.pctl.transforms.transforms.TargetTransform
│ _args_:
│ - ${dataset_description.classification_preprocessing_dict}
│ - ${dataset_description.classification_dict}
│ DropPointsByClass:
│ _target_: myria3d.pctl.transforms.transforms.DropPointsByClass
│ CopyFullPos:
│ _target_: myria3d.pctl.transforms.transforms.CopyFullPos
│ CopyFullPreparedTargets:
│ _target_: myria3d.pctl.transforms.transforms.CopyFullPreparedTargets
│ GridSampling:
│ _target_: torch_geometric.transforms.GridSampling
│ _args_:
│ - 0.25
│ MinimumNumNodes:
│ _target_: myria3d.pctl.transforms.transforms.MinimumNumNodes
│ _args_:
│ - 300
│ MaximumNumNodes:
│ _target_: myria3d.pctl.transforms.transforms.MaximumNumNodes
│ _args_:
│ - 40000
│ CopySampledPos:
│ _target_: myria3d.pctl.transforms.transforms.CopySampledPos
│ Center:
│ _target_: torch_geometric.transforms.Center
│ predict:
│ DropPointsByClass:
│ _target_: myria3d.pctl.transforms.transforms.DropPointsByClass
│ CopyFullPos:
│ _target_: myria3d.pctl.transforms.transforms.CopyFullPos
│ GridSampling:
│ _target_: torch_geometric.transforms.GridSampling
│ _args_:
│ - 0.25
│ MinimumNumNodes:
│ _target_: myria3d.pctl.transforms.transforms.MinimumNumNodes
│ _args_:
│ - 300
│ MaximumNumNodes:
│ _target_: myria3d.pctl.transforms.transforms.MaximumNumNodes
│ _args_:
│ - 40000
│ CopySampledPos:
│ _target_: myria3d.pctl.transforms.transforms.CopySampledPos
│ Center:
│ _target_: torch_geometric.transforms.Center
│ augmentations:
│ x_flip:
│ _target_: torch_geometric.transforms.RandomFlip
│ _args_:
│ - 0
│ p: 0.5
│ y_flip:
│ _target_: torch_geometric.transforms.RandomFlip
│ _args_:
│ - 1
│ p: 0.5
│ normalizations:
│ NullifyLowestZ:
│ _target_: myria3d.pctl.transforms.transforms.NullifyLowestZ
│ NormalizePos:
│ _target_: myria3d.pctl.transforms.transforms.NormalizePos
│ subtile_width: ${datamodule.subtile_width}
│ StandardizeRGBAndIntensity:
│ _target_: myria3d.pctl.transforms.transforms.StandardizeRGBAndIntensity
│ augmentations_list: '${oc.dict.values: datamodule.transforms.augmentations}'
│ preparations_train_list: '${oc.dict.values: datamodule.transforms.preparations.train}'
│ preparations_eval_list: '${oc.dict.values: datamodule.transforms.preparations.eval}'
│ preparations_predict_list: '${oc.dict.values: datamodule.transforms.preparations.predict}'
│ normalizations_list: '${oc.dict.values: datamodule.transforms.normalizations}'
│ _target_: myria3d.pctl.datamodule.hdf5.HDF5LidarDataModule
│ data_dir: null
│ split_csv_path: null
│ hdf5_file_path: /var/data/cgaydon/data/20220607_151_dalles_proto.myria3d-V2.4.hdf5
│ points_pre_transform:
│ _target_: functools.partial
│ _args_:
│ - ${get_method:myria3d.pctl.points_pre_transform.lidar_hd.lidar_hd_pre_transform}
│ pre_filter:
│ _target_: functools.partial
│ _args_:
│ - ${get_method:myria3d.pctl.dataset.utils.pre_filter_below_n_points}
│ min_num_nodes: 50
│ tile_width: 1000
│ subtile_width: 50
│ subtile_shape: square
│ subtile_overlap_train: 0
│ subtile_overlap_predict: ${predict.subtile_overlap}
│ batch_size: 2
│ num_workers: 3
│ prefetch_factor: 3
│
├── dataset_description
│ └── _convert_: all
│ classification_preprocessing_dict:
│ 3: 5
│ 4: 5
│ 160: 64
│ 161: 64
│ 162: 64
│ 0: 1
│ 7: 1
│ 46: 1
│ 47: 1
│ 48: 1
│ 49: 1
│ 50: 1
│ 51: 1
│ 52: 1
│ 53: 1
│ 54: 1
│ 55: 1
│ 56: 1
│ 57: 1
│ 58: 1
│ 64: 1
│ 66: 1
│ 67: 1
│ 77: 1
│ 155: 1
│ 204: 1
│ classification_dict:
│ 1: unclassified
│ 2: ground
│ 5: vegetation
│ 6: building
│ 9: water
│ 17: bridge
│ 64: lasting_above
│ class_weights:
│ - 0.25
│ - 0.1
│ - 0.1
│ - 0.5
│ - 2.0
│ - 2.0
│ - 2.0
│ d_in: 9
│ num_classes: 7
│
├── callbacks
│ └── log_code:
│ _target_: myria3d.callbacks.comet_callbacks.LogCode
│ code_dir: ${work_dir}/myria3d
│ log_logs_dir:
│ _target_: myria3d.callbacks.comet_callbacks.LogLogsPath
│ lr_monitor:
│ _target_: pytorch_lightning.callbacks.LearningRateMonitor
│ logging_interval: step
│ log_momentum: true
│ log_iou_by_class:
│ _target_: myria3d.callbacks.logging_callbacks.LogIoUByClass
│ classification_dict: ${dataset_description.classification_dict}
│ model_checkpoint:
│ _target_: pytorch_lightning.callbacks.ModelCheckpoint
│ monitor: val/loss_epoch
│ mode: min
│ save_top_k: 1
│ save_last: true
│ verbose: true
│ dirpath: checkpoints/
│ filename: epoch_{epoch:03d}
│ auto_insert_metric_name: false
│ early_stopping:
│ _target_: pytorch_lightning.callbacks.EarlyStopping
│ monitor: val/loss_epoch
│ mode: min
│ patience: 6
│ min_delta: 0
│
└── predict
└── src_las: tests/data/Sydney_dataset/newdata.las
output_dir: /tests/data/output_dir/
ckpt_path: /home/lqlnbcr/myria3d/trained_model_assets/proto151_V2.0_epoch_100_Myria3DV3.1.0.ckpt
subtile_overlap: 0
gpus: 0
interpolator:
_target_: myria3d.models.interpolation.Interpolator
interpolation_k: 10
classification_dict: ${dataset_description.classification_dict}
probas_to_save:
- building
- ground
predicted_classification_channel: confidence
entropy_channel: entropy
0it [00:00, ?it/s]
Hi @lqlnbcr,
Using python -m myria3d.predict
is deprecated so you were right to use python run.py task.task_name=predict
following the updated doc :)
It seems that the predict.output_dir=/tests/data/output_dir/
uses an absolute path that starts with "/", when you probably meant the relative "tests/data/output_dir/".
This alone would create an error as your command would not have writing credentials to create the /tests/ folder (this happens at the end of the inference here).
0it [00:00, ?it/s] comes from the tqdm here. This means you were at the 0th batch when the system crashed. If your las is small (it fits in a single batch) this is probably normal.
Feel free to make a small PR to make its meaning more explicit with e.g. unit=batches
as a tqdm parameter :)
Hi @CharlesGaydon , thank you for your reply. I found that code can't use relative paths, and it can be run by replacing all of relative paths with complete absolute paths. The results were excellent.
I'm using the latest version of the main branch myria3d. The las data dimensions I use for prediction include ['X','Y','Z','intensity','return_number','number_of_returns','synthetic','key_point','withheld','overlap','scanner_channel','scan_direction_flag','edge_of_flight_line','classification','user_data','scan_angle','point_source_id','gps_time','red','green','blue','Infrared'].
When I tried overfit on the toy dataset everything works fine with the code. But when I try to use the ckpt and yaml of "proto151_V2.0_epoch_100_Myria3DV3.1.0" in "trained_model_assets" for inference, an unknown error occurred. Terminates without any prompt.
My input: First:
python -m myria3d.predict --config-path /trained_model_assets/ --config-name proto151_V2.0_epoch_100_Myria3DV3.1.0_predict_config_V3.2.0.yaml predict.src_las=/tests/data/Sydney_dataset/newdata.las predict.output_dir=/tests/data/output_dir predict.ckpt_path=/trained_model_assets/proto151_V2.0_epoch_100_Myria3DV3.1.0.ckpt predict.gpus=0 datamodule.batch_size=2 hydra.run.dir=/tests/data/log_dir
Second:python run.py \ task.task_name=predict \ predict.src_las=tests/data/Sydney_dataset/newdata.las \ predict.output_dir=/tests/data/output_dir/ \ predict.gpus=0 \ datamodule.batch_size=2
The first input terminates without any return at all. The second input returns the complete config information and a string of characters "0it [00:00, ?it/s]", but I don't know what it means.
very much hope to get your help, thanks.