search

microsoft / scene_graph_benchmark

image scene graph generation benchmark
MIT License
380 stars 86 forks source link

How can I train without any problems? #64

Open sunn-u opened 2 years ago

sunn-u commented 2 years ago

I want to train the sg_network(reldn) with provided dataset(oi, vg) in this repo. So I downloaded datasets and used config files like sgg_configs/oi_vrd/R152FPN_reldn_oi.yaml. I only edit a directory of the dataset but I can't train the model.

I got nan losses on results.

2021-12-28 16:41:58,982 maskrcnn_benchmark.trainer INFO: eta: 31 days, 16:54:51 iter: 20 loss: nan (nan) loss_obj_classifier: 0.0000 (0.0000) loss_pred_classifier: nan (nan) loss_contrastive_sbj: nan (nan) loss_contrastive_obj: nan (nan) loss_so_contrastive_sbj: 0.0000 (nan) loss_so_contrastive_obj: 0.0000 (nan) loss_p_contrastive_sbj: 0.0000 (nan) loss_p_contrastive_obj: 0.0000 (nan) time: 0.5663 (16.9113) data: 0.0108 (0.0608) lr: 0.001800 max mem: 5621 INFO:maskrcnn_benchmark.trainer:eta: 31 days, 16:54:51 iter: 20 loss: nan (nan) loss_obj_classifier: 0.0000 (0.0000) loss_pred_classifier: nan (nan) loss_contrastive_sbj: nan (nan) loss_contrastive_obj: nan (nan) loss_so_contrastive_sbj: 0.0000 (nan) loss_so_contrastive_obj: 0.0000 (nan) loss_p_contrastive_sbj: 0.0000 (nan) loss_p_contrastive_obj: 0.0000 (nan) time: 0.5663 (16.9113) data: 0.0108 (0.0608) lr: 0.001800 max mem: 5621

I tried to fix this error but I can't handle it. How can I train without any problems?

ChenCongGit commented 2 years ago

Have you solved this problem?

LIUTAOGE commented 1 year ago

Have you solved it yet? Help