Questions about loss

[Jennieett]

Hi! I found that the loss of the first epoch has not decreased for a long time, and was always 9.000, is this reasonable? Here are some logs during training: 2023-04-21 22:56:43,536 - mmdet - INFO - Epoch [1][8050/28130] lr: 3.000e-04, eta: 14 days, 13:10:48, time: 2.615, data_time: 0.090, memory: 17458, loss_depth: 1.0000, loss_voxel_ce_c_0: 1.0000, loss_voxel_sem_scal_c_0: 1.0000, loss_voxel_geo_scal_c_0: 1.0000, loss_voxel_lovasz_c_0: 1.0000, loss_voxel_ce_fine: 1.0000, loss_voxel_sem_scal_fine: 1.0000, loss_voxel_geo_scal_fine: 1.0000, loss_voxel_lovasz_fine: 1.0000, loss: 9.0000, grad_norm: 9.1886 2023-04-21 22:58:51,067 - mmdet - INFO - Epoch [1][8100/28130] lr: 3.000e-04, eta: 14 days, 12:47:33, time: 2.551, data_time: 0.085, memory: 17458, loss_depth: 1.0000, loss_voxel_ce_c_0: 1.0000, loss_voxel_sem_scal_c_0: 1.0000, loss_voxel_geo_scal_c_0: 1.0000, loss_voxel_lovasz_c_0: 1.0000, loss_voxel_ce_fine: 1.0000, loss_voxel_sem_scal_fine: 1.0000, loss_voxel_geo_scal_fine: 1.0000, loss_voxel_lovasz_fine: 1.0000, loss: 9.0000, grad_norm: 10.1773 2023-04-21 23:01:01,270 - mmdet - INFO - Epoch [1][8150/28130] lr: 3.000e-04, eta: 14 days, 12:26:50, time: 2.604, data_time: 0.090, memory: 17458, loss_depth: 1.0000, loss_voxel_ce_c_0: 1.0000, loss_voxel_sem_scal_c_0: 1.0000, loss_voxel_geo_scal_c_0: 1.0000, loss_voxel_lovasz_c_0: 1.0000, loss_voxel_ce_fine: 1.0000, loss_voxel_sem_scal_fine: 1.0000, loss_voxel_geo_scal_fine: 1.0000, loss_voxel_lovasz_fine: 1.0000, loss: 9.0000, grad_norm: 10.2346 2023-04-21 23:03:09,146 - mmdet - INFO - Epoch [1][8200/28130] lr: 3.000e-04, eta: 14 days, 12:04:22, time: 2.557, data_time: 0.084, memory: 17458, loss_depth: 1.0000, loss_voxel_ce_c_0: 1.0000, loss_voxel_sem_scal_c_0: 1.0000, loss_voxel_geo_scal_c_0: 1.0000, loss_voxel_lovasz_c_0: 1.0000, loss_voxel_ce_fine: 1.0000, loss_voxel_sem_scal_fine: 1.0000, loss_voxel_geo_scal_fine: 1.0000, loss_voxel_lovasz_fine: 1.0000, loss: 9.0000, grad_norm: 10.6167 2023-04-21 23:05:19,048 - mmdet - INFO - Epoch [1][8250/28130] lr: 3.000e-04, eta: 14 days, 11:43:52, time: 2.598, data_time: 0.092, memory: 17458, loss_depth: 1.0000, loss_voxel_ce_c_0: 1.0000, loss_voxel_sem_scal_c_0: 1.0000, loss_voxel_geo_scal_c_0: 1.0000, loss_voxel_lovasz_c_0: 1.0000, loss_voxel_ce_fine: 1.0000, loss_voxel_sem_scal_fine: 1.0000, loss_voxel_geo_scal_fine: 1.0000, loss_voxel_lovasz_fine: 1.0000, loss: 9.0000, grad_norm: 10.9726 2023-04-21 23:07:30,623 - mmdet - INFO - Epoch [1][8300/28130] lr: 3.000e-04, eta: 14 days, 11:24:57, time: 2.631, data_time: 0.103, memory: 17458, loss_depth: 1.0000, loss_voxel_ce_c_0: 1.0000, loss_voxel_sem_scal_c_0: 1.0000, loss_voxel_geo_scal_c_0: 1.0000, loss_voxel_lovasz_c_0: 1.0000, loss_voxel_ce_fine: 1.0000, loss_voxel_sem_scal_fine: 1.0000, loss_voxel_geo_scal_fine: 1.0000, loss_voxel_lovasz_fine: 1.0000, loss: 9.0000, grad_norm: 10.8935

[JeffWang987]

Plz refer to https://github.com/JeffWang987/OpenOccupancy/issues/4