Got this output 'nter. Avg. Loss: nan' during training

[seven7777777]

Hi, I triained with run_inverse_cnn_train.sh like this: ---------------------------------shell script----------------------------

!/usr/bin/env bash

stylizedTrainDir=/data/notebook/dataset/annotations/small/train/stylized orgTrainDir=/data/notebook/dataset/annotations/small/train/original stylizedTestDir=/data/notebook/dataset/annotations/small/test/stylized orgTestDir=/data/notebook/dataset/annotations/small/test/original trainSet=/data/notebook/dataset/annotations/small/train/trainSetTxt valSet=/data/notebook/dataset/annotations/small/test/trainSetTxt.50 outputDir=models/small batchSize=8 nEpochs=12 gpu=0 uncertainty=aleatoric python main_inv_function.py --lr=0.002 --uncertainty=default --print-freq=2000 --gpu=$gpu --batchSize=$batchSize --outputDir=$outputDir --stylizedTrainDir=$stylizedTrainDir --orgTrainDir=$orgTrainDir --stylizedTestDir=$stylizedTestDir --orgTestDir=$orgTestDir --trainSet=$trainSet --valSet=$valSet --nEpochs=$nEpochs --uncertainty=${uncertainty}

but it keeps logout: ===> Epoch 1 Complete: Avg. Loss: nan, Seg Loss: 0.0000 ===> Avg. PSNR: nan dB ===> Avg. MSE: nan dB ===> Avg. Seg Loss: 0.0000 dB Checkpoint saved to model_epoch_1.pth ===> Epoch2: Inter. Avg. Loss: nan, Seg Loss: 0.0000

@jonghwa-yim Could you please give me some suggestions?

[jonghwa-yim]

It should work well if your dataset is correctly set. Or please try batchSize=16

[seven7777777]

It should work well if your dataset is correctly set. Or please try batchSize=16

Okay, thank you. I'll try. @jonghwa-yim