search

ywyue / RoomFormer

[CVPR 2023] RoomFormer: Two-level Queries for Single-stage Floorplan Reconstruction
https://ywyue.github.io/RoomFormer/
MIT License
163 stars 22 forks source link

Some advice on training the model #8

Closed PaulLiuYZ closed 1 year ago

PaulLiuYZ commented 1 year ago

Hi yuewen, thanks for your perfect work and congratulations that roomformer is accepted by CVPR! However, I have some problems trainning roomformer to get the good resluts, expecially with the angle precision and angle recall, and about 2 points lower than the results in the paper in the six evaluation metrics. Besides, in the code, the checkpoint is saved per 20 epoch, i wonder if there are any better way to save the checkpoints? Can you give me some advice on how to train the model? Thanks a lot if you can help me!

ywyue commented 1 year ago

Hi Yuzhou, thanks for the question! Could you please post your scores here? Based on my experience, the fluctuation of one point (± 1% ) is expected due to randomness in training. I will recheck the training and keep you posted in the next few days.

I only saved the checkpoint per 20 epochs for debugging purposes. We always report the scores using the checkpoint in the last epoch. An alternative way is to save the checkpoint based on the scores on the validation dataset, i.e. only save the checkpoint if the validation scores are higher than previous epochs. Hope it helps!

ywyue commented 1 year ago

I retrained one model, and here are the scores: image

The fluctuation of one point (± 1% ) is expected, but it is very unlikely that the results will be 2 points lower.

PaulLiuYZ commented 1 year ago

Hi yuewen, sorry for the late reply, i tried again later and i reproduced the results! Thank you again for your help!