[CVPR'22 Best Paper Finalist] Official PyTorch implementation of the method presented in "Learning Multi-View Aggregation In the Wild for Large-Scale 3D Semantic Segmentation"
Other
222
stars
24
forks
source link
My training is unable to execute the final Epoch. #31
While conducting the training, the code runs smoothly until the last Epoch, where it suddenly terminates without any error message. Subsequently, it prints out the training results.
Here is the translation of the provided output.log:
While conducting the training, the code runs smoothly until the last Epoch, where it suddenly terminates without any error message. Subsequently, it prints out the training results.
Here is the translation of the provided output.log:
100%|█| 2000/2000 [1:16:40<00:00, 2.30s/it, data_loading=0.087, iteration=1.525, train_acc=93.86, train_loss_cross_entropy=0 0%| | 0/2000 [00:00<?, ?it/s] [2023-07-31 06:48:07,126][torch_points3d.trainer][INFO] - Learning rate = 0.000800
100%|█| 2000/2000 [1:16:08<00:00, 2.28s/it, data_loading=0.071, iteration=1.029, train_acc=93.61, train_loss_cross_entropy=0 [2023-07-31 08:04:27,512][torch_points3d.trainer][INFO] - Learning rate = 0.000800 [2023-07-31 08:04:27,513][torch_points3d.trainer][INFO] - EPOCH 58 / 60
100%|█| 2000/2000 [1:16:17<00:00, 2.29s/it, data_loading=0.141, iteration=1.928, train_acc=93.56, train_loss_cross_entropy=0.208, train_loss_seg=0.208, train_m [2023-07-31 09:20:57,583][torch_points3d.trainer][INFO] - Learning rate = 0.000800 [2023-07-31 09:20:57,583][torch_points3d.trainer][INFO] - EPOCH 59 / 60
100%|█| 2000/2000 [1:16:15<00:00, 2.29s/it, data_loading=0.128, iteration=2.148, train_acc=93.69, train_loss_cross_entropy=0.198, train_loss_seg=0.198, train_m [2023-07-31 10:37:25,583][torch_points3d.trainer][INFO] - Learning rate = 0.000800