Open mancat49 opened 1 year ago
@mancat49 I suspect there won't be a way to do it from images. Being able to reverse the rasterization to the pose coordinates probably would be too big of an operation for a lightweight node like this one.
On the other hand, I could imagine this plugin getting that information directly from ComfyUI. comfyui_controlnet_aux (The currently best custom node for using the ControlNet preprocessors) posted a few ways of extracting the json data from a recent pose extraction.
The first way, I'd imagine would end up in this plugin itself at some point and be a huge improvement. If I was familiar enough with NodeJS, I'd make a pull request myself.
I did test a quick and dirty workaround based on the For API users: Python code.
To use it, you'd want to save this code in a .py file and run it with python after running the DWPreprocessor (an improved OpenPose extractor) node and filling in the value for the corresponding prompt_id inside the single quotes.
import json, urllib.request
server_address = "127.0.0.1:8188"
prompt_id = '' #Too lazy to POST /queue
output_file = "output/openpose.json"
def get_history(prompt_id):
with urllib.request.urlopen("http://{}/history/{}".format(server_address, prompt_id)) as response:
return json.loads(response.read())
history = get_history(prompt_id)[prompt_id]
for o in history['outputs']:
for node_id in history['outputs']:
node_output = history['outputs'][node_id]
if 'openpose_json' in node_output:
print(json.loads(node_output['openpose_json'][0]))
with open(output_file, "w") as f:
pose_info = json.loads(node_output['openpose_json'][0])
if not pose_info.get("width"):
pose_info["width"] = pose_info.get("canvas_width")
if not pose_info.get("height"):
pose_info["height"] = pose_info.get("canvas_height")
if not pose_info.get("keypoints"):
people = pose_info.get("people")
if not people or len(people) == 0:
raise RunTimeError("No people detected in openpose data")
keypoints = people[0].get("pose_keypoints_2d")
pose_info["keypoints"] = [[[keypoints[i], keypoints[i+1]] for i in range(0, len(keypoints), 3)]]
json.dump(pose_info, f)
print("Saved to", output_file)
With that, you'd at least be able to use the extracted poses as starting points. It would be missing a few extra details like finger positions. Those aren't supported in this custom node. But it would update every part of the pose that is currently supported in this repo.
Openpose Editor is a great addon in ComfyUI. But cannot retrieve the original pose from Openpose Recognition (png format) to modify it in the editor (json format).