search

purvaten / FLEX

Code for our CVPR'23 paper - "FLEX: Full-Body Grasping Without Full-Body Grasps"
https://flex.cs.columbia.edu/
105 stars 9 forks source link

Pretrained model vposer_grab.pt? #2

Closed siddharthKatageri closed 1 year ago

siddharthKatageri commented 1 year ago

Hi @purvaten By referring to the readme, I was able to download all the data and checkpoints except for vposer_grab.pt.

The flex.yaml says best_mnet_grab: 'flex/pretrained_models/ckpts/vposer_grab.pt' # Pre-trained HAND-GRASPING (trained on GRAB right-hand) . So I checked for it in GRAB website, but they only provide coarsenet.pt and refinenet.pt.

Where do I download vposer_grab.pt from??

Thanks!!

siddharthKatageri commented 1 year ago

I looked into the code, and in here, the VPoser model pre-trained on AMASS is loaded, as cfg.vposer_dset='amass'. So it's fine if there is no vposer_grab.pt.

But I had a doubt, please help me think through it, I went through the paper, and found that you use VPoser (for prior over full-body poses) and GrabNet (for prior over right hand grasps). If VPoser is for full-body prior, is using VPoser model pre-trained on GRAB (right hand) correct? Won't it fail to provide prior of full-body pose if that's the case?

purvaten commented 1 year ago

Hi @siddharthKatageri, good catch! vposer_grab.pt is indeed unnecessary and not used by the code. It was something I had created at some point -- it was a checkpoint of the VPoser model architecture but pre-trained on full-body data from GRAB only, not the rest of AMASS. Because GRAB-only poses are not as diverse as the rest of AMASS, it doesn't work great in practice.

I will remove it from the requirements.