Open j2l opened 1 year ago
Please @steve-salmond could you please explain how you get some ControlNet result? I'm pretty sure I'm doing something wrong here :smile:
Hmm, that's no good. Thanks for posting this. I'm not sure what's going wrong. While developing this feature, I tested with the depth model and one of the architectural ones, but I didn't actually test with openpose.
If I had to guess, I think there's a possibility that the preprocessor is the problem. I think the point of the preprocessor is to take a regular image (like one of a person) and then turn it into an image for the model to use (like a pose).
So maybe try without any preprocessor.
(I am traveling right now, without computer, so I won't be able to help much until later in the week. But feel free to send me more info on whatever you try!)
Any news?
Has anyone tried this while setting the preprocessor to none
?
Any news? AFAIK, prepocessor to none wouldn't treat the skeleton as a pose.
AFAIK, prepocessor to none wouldn't treat the skeleton as a pose.
I actually think that's exactly the way ControlNet works. With a preprocessor, it converts a regular image into a skeleton image. Without it, it uses the image input as the skeleton image.
I could be wrong - I haven't tested this with the openpose model. But I think this would be the best next step for you to try.
Ok, I see what you mean. Nope, it keeps on using the pose as image, not as pose:
I guess my process is wrong. Are you successful using a pose?
You're right about the preprocessor, in webui @benrugg , it is not needed.
t works only in webui, if I select Control Type "All" or "openpose":
But if I select Control Type as "canny":
So I guess, we're missing a selection of priority_order
in your code?
Ok, yeah, there must be a different way that Automatic1111's API is expecting the input. I will check it out and try to fix it when I have a chance. I am in the middle of a launch on a different (unrelated) product right now, so it might be a few weeks before I can dedicate time to this.
I understand @benrugg ... and wait :smile:
Would you please let me know about an ETA @benrugg ?
Hey, sorry, I can't give an ETA. Like I mentioned, I'm head down on a different project right now, so it will probably be a while. If you can find anyone else who can experiment with this, I'd be happy to review a PR.
Sure, but I need some little guidance first :smile:
Do you also think we miss priority_order
?
Quickly, do you have hints how to add it in the addon UI?
Thanks
I think it will likely take some experimentation to figure out what data and settings need to be passed to Automatic1111's API. My intuition is that it's not priority_order
, but it's possible. I think someone will need to dive into their API and my python code.
I can try to dive in, I'd just need to know how to add a setting like priority_order
in the UI.
I think for better or worse, the UI would probably be the last step in the process. Most of the work would be in python. Blender has a lot of guides, but it takes a lot to dive into it unless you already know how to code.
OK. Let me know when you tackle this one.
Describe the bug
Using a ControlNet Rig in Blender + AI render with ControlNet more recent control_v11p_sd15_openpose.pth (https://huggingface.co/lllyasviel/ControlNet-v1-1/tree/main) I get this funny result: (A future statue erected to AI maybe?)
My settings:
Any idea what I missed?
To reproduce
Install openpose from https://huggingface.co/lllyasviel/ControlNet-v1-1/tree/main add a openpose rig + mesh setup as above F12
Error log
none, but ControlNet openpose should work
Environment
Screenshots/video
No response
Additional information
No response