jin-qq
commented
11 months ago If I use https://github.com/facebookresearch/stable_signature/blob/main/utils_model.py#L134 to replace load_model_from_config , my other network have gradient and can be trained but stable-diffusion does not have gradient.
When I tried to finetune stable-diffusion, I import the load_model_from_config from scripts.txt2img, I found that gradient value equal to None even if require grad==True. When I import load_model_from_config, I create a MLP network (only one nn.Linear layer), this network do not have gradient as well. However, if I do not import load_model_from_config or any function from scripts.txt2img, MLP network have grandient and can be trained.