search

nie-lang / UnsupervisedDeepImageStitching

TIP2021 - Unsupervised deep image stitching network
311 stars 46 forks source link

Convergence of the learning curve and nan issue #23

Closed BoomSky0416 closed 2 years ago

BoomSky0416 commented 2 years ago

To implement this method into low overlap scenes, I finetuned the provided 100,0000-checkpoint on new dataset for low overlap scenes. But the loss curve doesn't converge. And If I train the model from scratch on the new dataset, the nan issue happened. Here are the learning curves for Finetune and From scratch. Do you have any methods to fix it? Is it related to the quality or size of new dataset? image image

nie-lang commented 2 years ago

Since this unsupervised work is based on a pixel-level alignment loss, the low overlap would significantly increase the difficulty of training such a network.

In my view, the new dataset should not only contain scenes of low overlap. It's suggested to include some scenes of higher overlap to balance the data distribution. Maybe, it can make the training process more stable. Or you can try a lower learning rate.

Btw, it seems the loss slightly decreased during your finetuning. Maybe you could try to train it on a synthetic dataset first. Then, you can fine-tune it on your own dataset.

BoomSky0416 commented 2 years ago

Thanks!