search

bharat-b7 / MultiGarmentNetwork

Repo for "Multi-Garment Net: Learning to Dress 3D People from Images, ICCV'19"
282 stars 65 forks source link

What does test_network.py actually do? #14

Closed neonb88 closed 4 years ago

neonb88 commented 4 years ago

Don't mean to be rude. I appreciate what documentation you've written. But it's still pretty unclear to me.

Does it fit the clothes in the test_data.pkl file to the SMPL approximation of the body also in test_data.pkl? If so, why are

  1. visualize_scan.py
  2. visualize_garments.py
  3. dress_SMPL.py

also necessary? I expect I will understand this well enough by the time you address this answer, but a high level description of each of the 4 .py files would still probably help future readers

===============================================

===============================================

===============================================

My current understanding is:

  1. visualize_scan.py allows us to create a SMPL model that best approximates the 1-8 images, PGN segmentations, and openpose keypoints, (and then dresses? right now my understanding is that visualize_scan.py doesn't do any clothes)
  2. visualize_garments.py doesn't allow us to swap in our own body (scan info)
  3. dress_SMPL.py: I'm not sure how this is different from visualize_garments.py.

I will update as I start to understand better. If anything, it will be useful notes for me, and hopefully for others

Cheers, Nathan

P.S. a lot of this is clarified by Mr. Bhatnagar in https://github.com/bharat-b7/MultiGarmentNetwork/issues/8#issuecomment-563178299.

bharat-b7 commented 4 years ago

Hi, Sorry for the confusion. I'll try to update the descriptions asap. For now here's a brief clarifications:

  1. visualize_scan.py, just loads a scan (not SMPL) and visualizes segmentation and texture on top. It has nothing to do with SMPL registration or garments.
  2. visualize_garments.py, visualizes 3D garments (from the digital wardrobe dataset) on their original SMPL body (with original pose and shape).
  3. dress_SMPL.py, allows to select 3D garments (from the digital wardrobe dataset) and dress any SMPL body (user can supply pose and shape parameters)

As for test_data.pkl, it contains 2D data that is required to run MGN on images. After you run MGN, you will get 3D garments and can use the scripts mentioned above for virtual dressing.

Hope this helps. Best, Bharat

neonb88 commented 4 years ago

Hi,

Thanks for your prompt reply. The whole pipeline is very impressive and obviously took a lot of time and effort.

In that case, I think dress_smpl.py currently fits my main needs

Cheers, Nathan

neonb88 commented 4 years ago

Oh also, I edited some of your code to work with python3.7, and when I ran test_network.py the clothing looked a little strange. I can send photos when I get back to my laptop. Should I do something to fix this?

Thanks again, Nathan

bharat-b7 commented 4 years ago

Thanks for the update. if you could share the images, we can take a look at what could be going wrong. A fix would be great!

neonb88 commented 4 years ago

Hi. At this point I don't remember the exact conditions under which I ran this, but here's the image anyway. Maybe it can help someone else mgn_failure_Feb_23_2020

bharat-b7 commented 4 years ago

This seems weird. Do you have the results for MGN prediction before the 2D refinement step?

neonb88 commented 4 years ago

@bharat-b7 Can address that later, but of more importance to me is a simple question: does this code recommend clothing that might fit a person (ie. given 1. a SMPL mesh and 2. a clothing mesh/registration/whatever you call it, can MGN output 3. a score predicting how good the clothes will fit on the person)

bharat-b7 commented 4 years ago

MGN predicts the 3D body (SMPL parameters) and garment meshes based on the input images. I'm afraid MGN cannot predict quality of fit.

neonb88 commented 4 years ago

Okay. Thank you

xiezhy6 commented 3 years ago

Hi, Sorry for the confusion. I'll try to update the descriptions asap. For now here's a brief clarifications:

  1. visualize_scan.py, just loads a scan (not SMPL) and visualizes segmentation and texture on top. It has nothing to do with SMPL registration or garments.
  2. visualize_garments.py, visualizes 3D garments (from the digital wardrobe dataset) on their original SMPL body (with original pose and shape).
  3. dress_SMPL.py, allows to select 3D garments (from the digital wardrobe dataset) and dress any SMPL body (user can supply pose and shape parameters)

As for test_data.pkl, it contains 2D data that is required to run MGN on images. After you run MGN, you will get 3D garments and can use the scripts mentioned above for virtual dressing.

Hope this helps. Best, Bharat

Hi, Thanks for your excellent work. I have just touched on this topic, so maybe my question is a little trivial. When I finished running the test_network.py, I can not find any result file in my directory. My question is How can I get the result file after running the test_network.py script and what datas should I save?

Thanks again! Zhenyu