search

facebookresearch / sapiens

High-resolution models for human tasks.
https://about.meta.com/realitylabs/codecavatars/sapiens/
Other
4.53k stars 258 forks source link

When the motion is too fast or the image quality is not clear .... #148

Closed chopin1998 closed 1 month ago

chopin1998 commented 1 month ago

When the motion is too fast or the image quality is not clear, the seg process will produce holes, leading to imperfect subsequent tasks.

https://github.com/user-attachments/assets/90ecb66c-c89e-4fd8-ba50-faf428b5133f

i use sapiens_1b for seg, 2b for normal (both bfloat 16)

Regarding this issue, can it be solved through fine-tuning?

Thank you!

rawalkhirodkar commented 1 month ago

@chopin1998 Can you try our binary segmentor? https://huggingface.co/facebook/sapiens-seg-foreground-1b-torchscript This is also used in all our demos. We found it be better than using class-wise segmentation model when the end goal is obtaining normal/depth.

Also, for normal estimation, removing background is not necessary, you can turn off the background removal process.

chopin1998 commented 1 month ago

@chopin1998 Can you try our binary segmentor? https://huggingface.co/facebook/sapiens-seg-foreground-1b-torchscript This is also used in all our demos. We found it be better than using class-wise segmentation model when the end goal is obtaining normal/depth.

Also, for normal estimation, removing background is not necessary, you can turn off the background removal process.

ok... i'll try. is there bfloat16 version?