search

lllyasviel / ControlNet-v1-1-nightly

Nightly release of ControlNet 1.1
4.48k stars 364 forks source link

How to finetune ? #43

Closed scarbain closed 1 year ago

scarbain commented 1 year ago

Hi everyone, thank you @lllyasviel for this wonderful tool you made !

I've been trying to finetune one of the latest V1.1 model : openpose.

I managed to modify the tutorial_train.py and tutorial_dataset.py from the original repo to make the training start but it doesn't seem to be following the openpose at all in the current early stages of training (< 10k steps).

I've tried to hack the tool_add_control and tool_transfer_control scripts to create a model with the latest of V1.1 ControlNet Openpose weights that I could feed to tutorial_train but it seems it's restarting from scratch anyway.

Can you provide some guidelines or feedback on how to finetune an existing model please ?

Thanks

lllyasviel commented 1 year ago

Thanks for the issue. see also the main repo for training doc.

Lanhaoran commented 1 year ago

Hey, I am also considering about this issue. Did you find some efficient method to fine-tune the model? Thanks!