search

facebookresearch / stable_signature

Official implementation of the paper "The Stable Signature Rooting Watermarks in Latent Diffusion Models"
Other
357 stars 48 forks source link

Question about Generate with Stability AI codebase #27

Open ypradhan222 opened 1 month ago

ypradhan222 commented 1 month ago

Hey author, Great Work but I am bit confused in this generate with Stability AI codebase. As you told to append the lines : state_dict = torch.load(path/to/ldm/checkpoint_000.pth)['ldm_decoder'] msg = model.first_stage_model.load_state_dict(state_dict, strict=False) print(f"loaded LDM decoder state_dict with message\n{msg}") print("you should check that the decoder keys are correctly matched") I appended it in txt2img file and ran it as python /working/another.py --prompt "a professional photograph of an astronaut riding a horse" --ckpt /working/v2-1_512-ema-pruned.ckpt --config /working/v2-inference.yaml --H 256 --W 256
but i am getting this error state_dict = torch.load('output/imgs/checkpoint_000.pth') loaded LDM decoder state_dict with message _IncompatibleKeys(missing_keys=['encoder.conv_in.weight', 'encoder.conv_in.bias', 'encoder.down.0.block.0.norm1.weight', 'encoder.down.0.block.0.norm1.bias', 'encoder.down.0.block.0.conv1.weight', 'encoder.down.0.block.0.conv1.bias', 'encoder.down.0.block.0.norm2.weight', 'encoder.down.0.block.0.norm2.bias', 'encoder.down.0.block.0.conv2.weight', 'encoder.down.0.block.0.conv2.bias', 'encoder.down.0.block.1.norm1.weight', 'encoder.down.0.block.1.norm1.bias', 'encoder.down.0.block.1.conv1.weight', 'encoder.down.0.block.1.conv1.bias', 'en.........) How to fix this ??

fenghe12 commented 3 weeks ago

have you solved this？

pierrefdz commented 3 weeks ago

Sorry I just saw the issue. Can you give me the full message (incompatible keys and missing keys)?

ypradhan222 commented 3 weeks ago

Sure here is the full message , (error.txt) Please correct me if I'm making any mistake in doing. Here is the file which I am executing , another.txt

pierrefdz commented 3 weeks ago

The first part of the error (L1 to L4) are due to the model loading and keys of the ckpts. You should run state_dict = torch.load(path/to/ldm/checkpoint_000.pth)['ldm_decoder'] as indicated in https://github.com/facebookresearch/stable_signature?tab=readme-ov-file#with-stability-ai-codebase. Note that is not an error, but a warning meaning that the LDM decoder that you were using was the original one.

The other part is due to problems with the diffuser library. I am not very familiar with it. My guess is that you should try to install a previous version of the library.

ypradhan222 commented 3 weeks ago

I mended the first part but still the same. Nothing changed.