search

ascust / 3DMM-Fitting-Pytorch

A 3DMM fitting framework using Pytorch.
594 stars 95 forks source link

How to get the face texture? #16

Closed sunshinewhy closed 2 years ago

sunshinewhy commented 2 years ago

Thank you for your awesome work! After running your project, I get the reconstructed face image and *.obj file, can you tell me how to get the rendered image just like these? 2

ascust commented 2 years ago

Please have a look at this line: https://github.com/ascust/3DMM-Fitting-Pytorch/blob/8ca87fc3ce42555d376dc505858e96536068896d/fit_single_img.py#L118 I guess this is what you are looking for. It is basically the output from the renderer of Pytorch3D, which is a 4 channel image (the forth channel is a mask). So simply convert it into a uint8 array and save it to an image type.

sunshinewhy commented 2 years ago

Please have a look at this line:

https://github.com/ascust/3DMM-Fitting-Pytorch/blob/8ca87fc3ce42555d376dc505858e96536068896d/fit_single_img.py#L118

I guess this is what you are looking for. It is basically the output from the renderer of Pytorch3D, which is a 4 channel image (the forth channel is a mask). So simply convert it into a uint8 array and save it to an image type.

Yes, you are right. You provide some images in data folder, and I want to know if I replace these images, should I retrain the model?

ascust commented 2 years ago

@sunshinewhy those are some sample images. It has nothing to do with model training. All the code does is "given an image or a video, the code is trying to find a set of parameters to describe the reconstructed image or the video."

sunshinewhy commented 2 years ago

@sunshinewhy those are some sample images. It has nothing to do with model training. All the code does is "given an image or a video, the code is trying to find a set of parameters to describe the reconstructed image or the video."

Nice, thanks a lot!

sunshinewhy commented 2 years ago

I use one RTX3080 GPU to render the image, each image cost about 35s (including 4s no-rigid render and 33s rigid render), I want to reduce the time. I find you provide the hyperparameters in the file, could you tell me how to set them, I will be grateful.

ascust commented 2 years ago

@sunshinewhy In the current version, I only use Adam as the optimizer to iteratively update variables, which might not be a fast way. To speed things up, one could replace the optimization method with some faster ones. I have not dug into it yet.

sunshinewhy commented 2 years ago

@ascust Okay, I reset the hyperparameters, it will save the time, but the results not good, maybe I should try other methods.