Can provide Normal model?

[bbk40221]

In train_Normal/demo_Normal.py, the model path is "/media/lx/4A42-E0B2/code_github/SePIFU_08_15into/checkpoints/model_Normal-Net/Train_Normal_epoch_27_1984",

netG = NormalNet(opt)
        netG.to(cuda)
        # 2023/05/06 we get the 27_1984
        model_path = '/media/lx/4A42-E0B2/code_github/SePIFU_08_15into/checkpoints/model_Normal-Net/Train_Normal_epoch_27_1984'
        netG.load_state_dict(torch.load(model_path, map_location=cuda))
        netG.eval()

This model can't be found in repository, can provide this model? Thanks.

[lxzlwhy]

Hi, because our paper is still under review, we will publish the normal model when the paper is about to be accepted. Please wait for a while. Thank you for your interest in our work.

[FernandoConti]

Hi, is there an update on the Normal model release? Thank you so much in advance!

[Megidd]

Any update on it?

Can we use such tools to create normal maps:

https://github.com/brycedrennan/imaginairy-normal-map

[FernandoConti]

Hi Megidd, why didn't that codebase work? Just curious :-)

[Megidd]

@FernandoConti I have shown below two normal maps:

• The left one is a sample created by the SEIF normal model.
• The right one is another sample created by https://github.com/brycedrennan/imaginairy-normal-map

As can be observed, the SEIF normal model - left one - creates much more fine details:

The right map creates a messed up 3D model eventually, maybe due to the normal map lacking the fine details:

[FernandoConti]

Thank you so much for showing me! Yes, much less detail and incorrect normals as well (e.g. cheeks).

Okay, so unfortunately we're not hearing anything from the creators, maybe we can train a new normal model ourselves. We need a library of highres head scans to train a new model, I'm going to research that.

[Megidd]

@FernandoConti That's right.

We believe that it might be possible to just train their normal model in some weeks. Considering the fact that SEIF is apparently using public datasets for their training.

[FernandoConti]

@Megidd Yes, I believe that as well. Unfortunately all 3D databases I found need academic approval.

Okay well, please keep me updated on your progress, I will do the same!

[leyuanCv]

Thank you for your interest in our work. We will upload the trained normal estimation model soon.

[Comfyux]

@leyuanCv Thank you for your response regarding the normal estimation model. We understand that you will release the trained model soon, and we greatly appreciate your efforts in sharing this valuable resource with the community.

If possible, could you please provide an estimated timeline for when we can expect the model release? This information will help us plan and prepare accordingly for integrating the model into our workflow.

We are eager to explore and utilize this advanced normal estimation model, and we look forward to your update. Once again, thank you for your contribution to the field.

[Comfyux]

@FernandoConti Thanks for your interest in training a new normal model. I noticed you mentioned researching to find a library of high-res head scans to train on. If you haven't found a suitable dataset yet, I may be able to provide some training data. I have access to a collection of 3D face scans that could potentially work.

If you would be interested in collaborating and want to discuss further, feel free to reach out to me at qu1989x@gmail.com . I'd be happy to share more details about the data I have available. Combining our efforts, we may be able to make progress on training an improved normal estimation model.

Please let me know if you would like to explore this possibility together. I look forward to hearing from you.

[FernandoConti]

Hello @Comfyux , thank you for reaching out to me!

The normal estimation model download link has been added to this repo, but it is impossible for us to download it, we cannot access the Baidu network.

Would you maybe be able to help us?

[nvdomidi]

@FernandoConti

I was able to download the file. I've shared it here for others who don't have access to baidu.

[FernandoConti]

@nvdomidi Thank you so much!

[Megidd]

Approach

We tested by these steps:

1. Cloned this repo on master branch.
2. Created a checkpoints folder in the main directory, then pasted the downloaded normal estimate model Train_Normal_epoch_27_1984 into that folder.
3. Took our custom image and renamed it to 1.png and 1_mask.png and put them into the image folder.
4. Ran the demo.py and got 1_normal.png.
5. Pasted 1.png, 1_mask.png and 1_normal.png into SeIF's demo folder.
6. Ran main_test.py.

Result

Our result is shown below. @FernandoConti, I'm just curious, what do your results look like?

[FernandoConti]

Hi @Megidd, we have similar results, they are pretty bad. Apparently the creators have been cherry picking their results...

[leyuanCv]

Hi @Megidd, we have similar results, they are pretty bad. Apparently the creators have been cherry picking their results...

Could you please share your input images so we can investigate the cause? We have tested up to 25,000 front-facing images without glasses. It is disheartening to receive such feedback.