search

shunsukesaito / PIFu

This repository contains the code for the paper "PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization"
https://shunsukesaito.github.io/PIFu/
Other
1.76k stars 341 forks source link

Training Result #44

Closed Brawie closed 4 years ago

Brawie commented 4 years ago

Hello shunsuke, I trained a netG using my own data, but the reconstructed meshes are not as good as the result of your released model. I'm not sure whether it is the data problem or did I miss something. Can you show some details like how is the MSE error going during your training process? Thank you so much!

Brawie commented 4 years ago

A typical result looks like this: Screenshot from 2020-07-09 14-29-57

shunsukesaito commented 4 years ago

How many stacks are you using? Depending on your data size, you may need to train longer for convergence. I also observed significant error drop when learning rate decays, so please make sure you set proper number of epochs and decay timing with RMSProps.

Another thing you could take a look is how you align your training data. If the depth center is not very consistent across data samples, the results could be deteriorated. If you have access to underlining skeleton information, probably you can align the depth center of each object with pelvis joint (but notice that you need to modify the code to accommodate this modification).

Brawie commented 4 years ago

I'm using 4 stacks as default setting. My 3d data doesn't contain skeleton informations, it's just a obj file. I have rescaled and translated them to the same height and same center. But the digital humans face different directions, will this have an impact? During training, the train set error stop at 0.05 and test set error stop at 0.15, is that good enough? I will try to change the learning rate decay timing. Thank you for your caring feed back!

shunsukesaito commented 4 years ago

But the digital humans face different directions, will this have an impact?

PIFu is also trained with all 360 degrees, so training with multiple direction should not degrade reconstruction too much.

During training, the train set error stop at 0.05 and test set error stop at 0.15, is that good enough?

It'd be difficult to provide useful feedback on this with custom data. I'd suggest training longer with proper learning rate decay to see if you get any better number.

Another thing is you may want to make sure your input meshes are watertight. Otherwise, the occupancy computation is likely incorrect and it hurts the performance.

Brawie commented 4 years ago

Ok, thank you for your suggestion!~