loss:nan怎么解决？

C:\F\pycharm2020.2\PytorchOCR-master>python tools/rec_train.py 2023-01-26 20:49:51,783 - torchocr - INFO - {'exp_name': 'CRNN', 'train_options': {'resume_from': '', 'third_party_name': '', 'checkpoint_save_dir': 'C:/Users/Public/pytorch/CRNN/checkpoint', 'device': 'cuda:0', 'epochs': 200, 'fine_tune_stage': ['backbone', 'neck', 'head'], 'print_interval': 10, 'val_interval': 2000, 'ckpt_save_type': 'HighestAcc', 'ckpt_save_epoch': 4}, 'SEED': 927, 'optimizer': {'type': 'Adam', 'lr': 0.001, 'weight_decay': 0.0001}, 'lr_scheduler': {'type': 'StepLR', 'step_size': 60, 'gamma': 0.5}, 'model': {'type': 'RecModel', 'backbone': {'type': 'MobileNetV3', 'model_name': 'small'}, 'neck': {'type': 'PPaddleRNN', 'hidden_size': 48}, 'head': {'type': 'CTC', 'n_class': 63465}, 'in_channels': 3}, 'loss': {'type': 'CTCLoss', 'blank_idx': 0}, 'dataset': {'alphabet': 'C:\F\japan_dict.txt', 'train': {'dataset': {'type': 'RecTextLineDataset', 'file': 'C:\F\pytorch_labels_combined.txt', 'input_h': 32, 'mean': 0.5, 'std': 0.5, 'augmentation': False}, 'loader': {'type': 'DataLoader', 'batch_size': 16, 'shuffle': True, 'num_workers': 1, 'collate_fn': {'type': 'RecCollateFn', 'img_w': 120}}}, 'eval': {'dataset': {'type': 'RecTextLineDataset', 'file': 'C:\F\2cleancn-20000 - 副本.txt', 'input_h': 32, 'mean': 0.5, 'std': 0.5, 'augmentation': False}, 'loader': {'type': 'RecDataLoader', 'batch_size': 4, 'shuffle': False, 'num_workers': 1, 'collate_fn': {'type': 'RecCollateFn', 'img_w': 120}}}}} 2023-01-26 20:49:51,890 - torchocr - INFO - net resume from scratch. 2023-01-26 20:50:00,587 - torchocr - INFO - Training... 2023-01-26 20:50:00,587 - torchocr - INFO - train dataset has 3615327 samples,225958 in dataloader 2023-01-26 20:50:00,588 - torchocr - INFO - eval dataset has 20000 samples,5000 in dataloader 2023-01-26 20:50:34,076 - torchocr - INFO - [0/200] - [10/225958] - lr:0.001 - loss:147.4167 - acc:0.0000 - norm_edit_dis:0.0000 - time:33.4870 2023-01-26 20:51:04,685 - torchocr - INFO - [0/200] - [20/225958] - lr:0.001 - loss:271.0975 - acc:0.0000 - norm_edit_dis:0.0000 - time:30.6092 2023-01-26 20:51:33,723 - torchocr - INFO - [0/200] - [30/225958] - lr:0.001 - loss:125.6765 - acc:0.0000 - norm_edit_dis:0.0000 - time:29.0379 2023-01-26 20:52:03,072 - torchocr - INFO - [0/200] - [40/225958] - lr:0.001 - loss:104.1359 - acc:0.0000 - norm_edit_dis:0.0000 - time:29.3491 2023-01-26 20:52:31,204 - torchocr - INFO - [0/200] - [50/225958] - lr:0.001 - loss:66.1327 - acc:0.0000 - norm_edit_dis:0.0000 - time:28.1294 2023-01-26 20:53:04,027 - torchocr - INFO - [0/200] - [60/225958] - lr:0.001 - loss:41.0694 - acc:0.0000 - norm_edit_dis:0.0000 - time:32.8223 2023-01-26 20:53:35,120 - torchocr - INFO - [0/200] - [70/225958] - lr:0.001 - loss:22.6170 - acc:0.0000 - norm_edit_dis:0.0000 - time:31.0934 2023-01-26 20:54:06,949 - torchocr - INFO - [0/200] - [80/225958] - lr:0.001 - loss:11.2823 - acc:0.0000 - norm_edit_dis:0.0000 - time:31.8285 2023-01-26 20:54:42,005 - torchocr - INFO - [0/200] - [90/225958] - lr:0.001 - loss:8.6058 - acc:0.0000 - norm_edit_dis:0.0000 - time:35.0553 2023-01-26 20:55:11,051 - torchocr - INFO - [0/200] - [100/225958] - lr:0.001 - loss:nan - acc:0.0000 - norm_edit_dis:0.0000 - time:29.0454 2023-01-26 20:55:41,430 - torchocr - INFO - [0/200] - [110/225958] - lr:0.001 - loss:nan - acc:0.0000 - norm_edit_dis:0.0000 - time:30.3788 2023-01-26 20:56:12,583 - torchocr - INFO - [0/200] - [120/225958] - lr:0.001 - loss:nan - acc:0.0000 - norm_edit_dis:0.0000 - time:31.1517 2023-01-26 20:56:43,814 - torchocr - INFO - [0/200] - [130/225958] - lr:0.001 - loss:nan - acc:0.0000 - norm_edit_dis:0.0000 - time:31.2295 2023-01-26 20:57:16,205 - torchocr - INFO - [0/200] - [140/225958] - lr:0.001 - loss:nan - acc:0.0000 - norm_edit_dis:0.0000 - time:32.3914 2023-01-26 20:57:46,468 - torchocr - INFO - [0/200] - [150/225958] - lr:0.001 - loss:nan - acc:0.0000 - norm_edit_dis:0.0000 - time:30.2631 2023-01-26 20:58:21,284 - torchocr - INFO - [0/200] - [160/225958] - lr:0.001 - loss:nan - acc:0.0000 - norm_edit_dis:0.0000 - time:34.8160 2023-01-26 20:58:52,547 - torchocr - INFO - [0/200] - [170/225958] - lr:0.001 - loss:nan - acc:0.0000 - norm_edit_dis:0.0000 - time:31.2620 2023-01-26 20:59:25,394 - torchocr - INFO - [0/200] - [180/225958] - lr:0.001 - loss:nan - acc:0.0000 - norm_edit_dis:0.0000 - time:31.1438 2023-01-26 20:59:58,949 - torchocr - INFO - [0/200] - [190/225958] - lr:0.001 - loss:nan - acc:0.0000 - norm_edit_dis:0.0000 - time:33.5533 2023-01-26 21:00:27,844 - torchocr - INFO - [0/200] - [200/225958] - lr:0.001 - loss:nan - acc:0.0000 - norm_edit_dis:0.0000 - time:28.8958

WenmuZhou / PytorchOCR

loss:nan怎么解决？ #260