search

eth-ait / MultiPly

MultiPly: Reconstruction of Multiple People from Monocular Video in the Wild (CVPR2024 Oral)
https://eth-ait.github.io/MultiPly/
174 stars 14 forks source link

Single figure preprocessing is broken #5

Open atonalfreerider opened 2 weeks ago

atonalfreerider commented 2 weeks ago

To reproduce: set run_preprocessing_multiply.sh number=1 # number of people

Proposed changes: multiply_model.py

assert batch_inputs["smpl_params"][:, 0, 0] == batch_inputs["smpl_params"][:, 0, 0] #TODO is the 1 index supposed to refer to the second person in the tracking?

sam_model.py:

if image_mask_all.shape[0] < 2:
                # TODO not sure if mask needs more processing before it is appended
                output_mask_list.append(image_mask_all[0])
                continue

See changes in this forked branch: https://github.com/eth-ait/MultiPly/compare/main...atonalfreerider:MultiPly:bugfix-1-person

jzr99 commented 1 week ago

Hi,

Our method, MultiPly, mainly focuses on the reconstruction of Multiple people. We have not tested it on single-person reconstruction for now. The prompting for SAM (especially for the negative point prompt) needs to be adjusted. I'll look into it in the future.