search

noahcao / Pixel2Mesh

A complete Pixel2Mesh implementation in PyTorch
230 stars 39 forks source link

Question about the training time? #24

Closed xiehousen closed 4 years ago

xiehousen commented 4 years ago

Hello, I used your code to train ShapeNet dataset, but I have been training for 8 days, and the effect is still poor.

I want to ask how long you have trained to get good results(like resnet.pth.tar).

Thank you very much for your reply.

ultmaster commented 4 years ago

Hi. It takes 4-5 days on 8 1080Ti GPUs. Hope it helps.

xiehousen commented 4 years ago

Hi. It takes 4-5 days on 8 1080Ti GPUs. Hope it helps.

I have a problem in training. I use four Titan XP for training, set the Batchsize to 8, and use the RESNET model provided by you as the pre-training model. However, after a few days of training, its CD is higher than the results you provide, and it is much higher, and it is very unstable during the training process. I really don't know what went wrong.

noahcao commented 4 years ago

Hi @xiehousen,

to have a better sense of your problem. Could you provide some log and settings of your training process, e.g., eval curve, loss curve, learning rate settings and so on.

noahcao commented 4 years ago

As there has been no follow-up for long, I'm closing this issue. Please feel free to reopen it if anyone wants to discuss about the training time.