search

lijiaman / CASENet

30 stars 2 forks source link

Loss extremely high and doesn't decrease #9

Open HoudaC opened 4 years ago

HoudaC commented 4 years ago

Hello, I follow your code and run it on a subset of SBD database (50 images). The code run til the end but the loss doesn't go down. The loss also is extremely high (about 1000000). Is it normal?

I run it with the following configuration: config:Namespace(batch_size=1, checkpoint_folder='./checkpoint', cls_num=20, ep\ ochs=150, lr=1e-10, lr_steps=[10000, 20000, 30000, 40000], momentum=0.9, multig\ pu=False, pretrained_model='', print_freq=1, resume_model='', start_epoch=0, we\ ight_decay=0.0005, workers=16)

Looking forward to your reply

gntoni commented 4 years ago

Make sure to change: model.eval()

for:

    model.train()
    optimizer.zero_grad()

in: https://github.com/lijiaman/CASENet/blob/0aa475d856ebaa5fa7bbb1a144da606c9ad4198c/train_val/model_play.py#L28