Plan for adding support on mediapipe holistic for pose+hands+face mesh?

[p0mad]

Hi, I was wondering if i can use Mediapipe holistics ( which includes 543 whole body keypoints) to the controlnet or not?

I saw that openpose keypoints can be added and used as a guided stype but it has some limitations and also a lot less keypoints, thus the output would not be the desired style!

Thanks Best regards

[geroldmeisinger]

I don't know "Mediapipe holistics" but you probably can if train your own control net. you would need an image dataset about humans (search for laion humans) and a pose estimator which produces the "Mediapipe holistics" model.

I wrote a long article about control net training which might help https://civitai.com/articles/2078

you might also be interest in control net 1.1 openpose https://github.com/lllyasviel/ControlNet-v1-1-nightly#controlnet-11-openpose and dense pose https://civitai.com/models/120149/controlnet-for-densepose and if you search you might also find other pose control net models (pifpaf for example)

[FurkanGozukara]

I don't know "Mediapipe holistics" but you probably can if train your own control net. you would need an image dataset about humans (search for laion humans) and a pose estimator which produces the "Mediapipe holistics" model.

I wrote a long article about control net training which might help https://civitai.com/articles/2078

you might also be interest in control net 1.1 openpose https://github.com/lllyasviel/ControlNet-v1-1-nightly#controlnet-11-openpose and dense pose https://civitai.com/models/120149/controlnet-for-densepose and if you search you might also find other pose control net models (pifpaf for example)

hello. could you help me with converting a video into DensePose version?

I have been struggling for a while atm

https://github.com/facebookresearch/detectron2/issues/5170