3D mesh is not generated

microsoft / MeshTransformer

Research code for CVPR 2021 paper "End-to-End Human Pose and Mesh Reconstruction with Transformers"

https://arxiv.org/abs/2012.09760

MIT License

614 stars 95 forks source link

3D mesh is not generated #15

Closed tqtrunghnvn closed 3 years ago

tqtrunghnvn commented 3 years ago

Hi,

First of all, thank you so much for your contribution!

I am facing a problem when running your quick demo code. I hope 3D mesh is generated as shown in your demo.

However, 3D mesh was not generated in my example.

Could you please give me some advice?

kevinlin311tw commented 3 years ago

Could you send your test sample to me? I can try running it.

tqtrunghnvn commented 3 years ago

I simply downloaded an image from the internet. You can try with this following one: https://keyassets.timeincuk.net/inspirewp/live/wp-content/uploads/sites/64/2018/12/champ-630x473.jpg

kevinlin311tw commented 3 years ago

Three more questions:

Are you able to generate our 3D mesh examples? This is to make sure you have install the code correctly.
Before feeding your golf image, do you crop the image with a size of 224x224? We need 224x224 image size.
Could you please share any error messages you got here?

tqtrunghnvn commented 3 years ago

Hi, I was not able to generate the 3D mesh for your examples. I got: 3dpw_test1_metro_pred 3dpw_test2_metro_pred 3dpw_test3_metro_pred

Could you please try to run another sample?

kevinlin311tw commented 3 years ago

You may find it here.

champ_metro_pred

In your attached results, I see you have generated the attention visualization with the output vertices. This indicates 3D mesh is generated. The issue you got is probably a rendering issue. I would suggest to double check the installation, making sure OpenDR is successfully installed.

tqtrunghnvn commented 3 years ago

Hmm. Thank you for your effort!

I have double-checked opendr installation, and is successfully installed and could be imported without any problems. Could you tell me where is the code for generating 3D mesh (NOT attention visualization)?

kevinlin311tw commented 3 years ago

I guess you are referring to rendering 3D mesh, right? You may check it here: https://github.com/microsoft/MeshTransformer/blob/44760db076e4dd1220c907b2b01a8b0e8c3a334c/metro/utils/renderer.py#L405

Basically it takes mesh vertices, camera parameters, input image as inputs. It will render 3D pink mesh on top of the image.

tqtrunghnvn commented 3 years ago

Is it mandatory to install the dependencies with the following versions?

Python 3.7
Pytorch 1.4
torchvision 0.5.0
cuda 10.1

kevinlin311tw commented 3 years ago

We verified that our code works with the dependencies. I would suggest to literally follow our installation instruction, and make sure you can reproduce our demo examples.

tqtrunghnvn commented 3 years ago

Thank you so much! The problem is solved.

zws2lll commented 3 years ago

@tqtrunghnvn hi, tqtrunghnvn, could you tell me how to solve this problem? I run the demo on gpu 3090.

tqtrunghnvn commented 3 years ago

Hi @zws2lll , you should check your opendr installation.

zws2lll commented 3 years ago

@tqtrunghnvn I use "pip install opendr", and it's 0.78. There's no error. How do you check the opendr?

Successfully built opendr Installing collected packages: opendr Attempting uninstall: opendr Found existing installation: opendr 0.76 Uninstalling opendr-0.76: Successfully uninstalled opendr-0.76 Successfully installed opendr-0.78

tqtrunghnvn commented 3 years ago

You could try: https://gitlab.eecs.umich.edu/ngv-python-modules/opendr

zws2lll commented 3 years ago

I clone it and "pip install .". 0.76 is installed. The output contains no render hand

./metro/tools/end2end_inference_handmesh.py --resume_checkpoint ./models/metro_release/metro_hand_state_dict.bin --image_file_or_path ./samples/hand/ freihand_sample3_metro_pred

zws2lll commented 3 years ago

I clone it and "pip install .". 0.76 is installed. The output contains no render hand

./metro/tools/end2end_inference_handmesh.py --resume_checkpoint ./models/metro_release/metro_hand_state_dict.bin --image_file_or_path ./samples/hand/

@kevinlin311tw hi, kevinlin311tw. I also use your docker image. Same result with no render hand. Any Suggestion？

tqtrunghnvn commented 3 years ago

@zws2lll I also used opendr0.76. Sorry I have no idea now

zws2lll commented 3 years ago

Thank you anyway.

fmx789 commented 2 years ago

Thank you anyway.

@zws2lll Hi, have you solved the problem? Such a circumstance perplexes me as well.

JCheng314 commented 2 months ago

For hand, I pulled relevant codes for generating mesh on the images and stored the vertices, camera data, then did the mesh generation in a separate file, and mesh is generated, but I don't know why the almost exactly the same code doesn't work.