search

autonomousvision / occupancy_networks

This repository contains the code for the paper "Occupancy Networks - Learning 3D Reconstruction in Function Space"
https://avg.is.tuebingen.mpg.de/publications/occupancy-networks
MIT License
1.51k stars 292 forks source link

Validation Loss Increasing #107

Open JiangWenPL opened 2 years ago

JiangWenPL commented 2 years ago

Hi authors, thank you for your great work.

I tried to train your network from scratch with ./configs/img/onet.yaml. I find something weird that the validation loss is increasing during training though the validation IOU is getting better. I also evaluated the model and the result looks fine(0.584 mean IOU). I am wondering if the validation loss would be normal.

Here is the screen shot for the loss curves. image

fangchuan commented 1 year ago

@JiangWenPL Hi, have you solved the problem? I found same problems when i finetune the pretrained model on iGibson dataset, the training loss is declining while the validation loss is rising, though the validation IOU seems fine. At begining, I thought the network was overfitting, then I stop the iteration as early as possible, but it did little a help the evaluation result in test dataset.