FusionBrainLab / StyleFeatureEditor

Official Implementation for "The Devil is in the Details: StyleFeatureEditor for Detail-Rich StyleGAN Inversion and High Quality Image Editing"
MIT License
98 stars 11 forks source link

Question about 'sfe_editor_light.pt' #3

Closed LiXm1002 closed 2 months ago

LiXm1002 commented 2 months ago

Thanks for releasing code! I want to run the code to edit my images, but some error reported. First, I create a simple_demo.py in runner folder, content as:

import sys sys.path = ['.'] + sys.path from simple_runner import SimpleRunner

runner = SimpleRunner( editor_ckpt_pth="pretrained_models/sfe_editor_light.pt" ) print(runner.available_editings())

then I run the file, it shows: File "/home/ubuntu/miniconda3/envs/recon/lib/python3.11/dataclasses.py", line 815, in _get_field raise ValueError(f'mutable default {type(f.default)} for field ' ValueError: mutable default <class 'configs.paths.DefaultPathsClass'> for field paths is not allowed: use default_factory

I modify dataclasses.field(default=v.default) to dataclasses.field(default_factory=v.default) in class_registry.py, line 29.

And another error reported as: Loading default Discriminator from pretrained_models/stylegan2-ffhq-config-f.pkl Loading from checkpoint: pretrained_models/sfe_editor_light.pt Traceback (most recent call last): File "/home/ubuntu/Documents/StyleFeatureEditor-main/runners/simple_demo.py", line 3, in <module> from simple_runner import SimpleRunner File "/home/ubuntu/Documents/StyleFeatureEditor-main/runners/simple_runner.py", line 188, in <module> runner = SimpleRunner( ^^^^^^^^^^^^^ File "/home/ubuntu/Documents/StyleFeatureEditor-main/runners/simple_runner.py", line 86, in __init__ self.inference_runner.setup() File "/home/ubuntu/Documents/StyleFeatureEditor-main/runners/base_runner.py", line 25, in setup self._setup_method() File "/home/ubuntu/Documents/StyleFeatureEditor-main/runners/base_runner.py", line 93, in _setup_method self.method = methods_registry[method_name]( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/ubuntu/Documents/StyleFeatureEditor-main/models/methods.py", line 52, in __init__ self.load_weights() File "/home/ubuntu/Documents/StyleFeatureEditor-main/models/methods.py", line 73, in load_weights self.discriminator.load_state_dict(get_keys(ckpt, "discriminator"), strict=True) File "/home/ubuntu/miniconda3/envs/recon/lib/python3.11/site-packages/torch/nn/modules/module.py", line 2153, in load_state_dict raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format( RuntimeError: Error(s) in loading state_dict for Discriminator: Missing key(s) in state_dict: "b1024.resample_filter", "b1024.fromrgb.weight", "b1024.fromrgb.bias", "b1024.fromrgb.resample_filter", "b1024.conv0.weight", "b1024.conv0.bias", "b1024.conv0.resample_filter", "b1024.conv1.weight", "b1024.conv1.bias", "b1024.conv1.resample_filter", "b1024.skip.weight", "b1024.skip.resample_filter", "b512.resample_filter", "b512.conv0.weight", "b512.conv0.bias", "b512.conv0.resample_filter", "b512.conv1.weight", "b512.conv1.bias", "b512.conv1.resample_filter", "b512.skip.weight", "b512.skip.resample_filter", "b256.resample_filter", "b256.conv0.weight", "b256.conv0.bias", "b256.conv0.resample_filter", "b256.conv1.weight", "b256.conv1.bias", "b256.conv1.resample_filter", "b256.skip.weight", "b256.skip.resample_filter", "b128.resample_filter", "b128.conv0.weight", "b128.conv0.bias", "b128.conv0.resample_filter", "b128.conv1.weight", "b128.conv1.bias", "b128.conv1.resample_filter", "b128.skip.weight", "b128.skip.resample_filter", "b64.resample_filter", "b64.conv0.weight", "b64.conv0.bias", "b64.conv0.resample_filter", "b64.conv1.weight", "b64.conv1.bias", "b64.conv1.resample_filter", "b64.skip.weight", "b64.skip.resample_filter", "b32.resample_filter", "b32.conv0.weight", "b32.conv0.bias", "b32.conv0.resample_filter", "b32.conv1.weight", "b32.conv1.bias", "b32.conv1.resample_filter", "b32.skip.weight", "b32.skip.resample_filter", "b16.resample_filter", "b16.conv0.weight", "b16.conv0.bias", "b16.conv0.resample_filter", "b16.conv1.weight", "b16.conv1.bias", "b16.conv1.resample_filter", "b16.skip.weight", "b16.skip.resample_filter", "b8.resample_filter", "b8.conv0.weight", "b8.conv0.bias", "b8.conv0.resample_filter", "b8.conv1.weight", "b8.conv1.bias", "b8.conv1.resample_filter", "b8.skip.weight", "b8.skip.resample_filter", "b4.conv.weight", "b4.conv.bias", "b4.conv.resample_filter", "b4.fc.weight", "b4.fc.bias", "b4.out.weight", "b4.out.bias".

I'm trying to print some information in methods.py, line 72, it shows: ipdb> self.encoder.load_state_dict(get_keys(ckpt, "encoder"), strict=True) <All keys matched successfully> ipdb> self.inverter.load_state_dict(get_keys(ckpt, "inverter"), strict=True) <All keys matched successfully> ipdb> self.discriminator.load_state_dict(get_keys(ckpt, "discriminator"), strict=True) *** RuntimeError: Error(s) in loading state_dict for Discriminator: Missing key(s) in state_dict: "b1024.resample_filter", "b1024.fromrgb.weight", "b1024.fromrgb.bias", "b1024.fromrgb.resample_filter", "b1024.conv0.weight", "b1024.conv0.bias", "b1024.conv0.resample_filter", "b1024.conv1.weight", "b1024.conv1.bias", "b1024.conv1.resample_filter", "b1024.skip.weight", "b1024.skip.resample_filter", "b512.resample_filter", "b512.conv0.weight", "b512.conv0.bias", "b512.conv0.resample_filter", "b512.conv1.weight", "b512.conv1.bias", "b512.conv1.resample_filter", "b512.skip.weight", "b512.skip.resample_filter", "b256.resample_filter", "b256.conv0.weight", "b256.conv0.bias", "b256.conv0.resample_filter", "b256.conv1.weight", "b256.conv1.bias", "b256.conv1.resample_filter", "b256.skip.weight", "b256.skip.resample_filter", "b128.resample_filter", "b128.conv0.weight", "b128.conv0.bias", "b128.conv0.resample_filter", "b128.conv1.weight", "b128.conv1.bias", "b128.conv1.resample_filter", "b128.skip.weight", "b128.skip.resample_filter", "b64.resample_filter", "b64.conv0.weight", "b64.conv0.bias", "b64.conv0.resample_filter", "b64.conv1.weight", "b64.conv1.bias", "b64.conv1.resample_filter", "b64.skip.weight", "b64.skip.resample_filter", "b32.resample_filter", "b32.conv0.weight", "b32.conv0.bias", "b32.conv0.resample_filter", "b32.conv1.weight", "b32.conv1.bias", "b32.conv1.resample_filter", "b32.skip.weight", "b32.skip.resample_filter", "b16.resample_filter", "b16.conv0.weight", "b16.conv0.bias", "b16.conv0.resample_filter", "b16.conv1.weight", "b16.conv1.bias", "b16.conv1.resample_filter", "b16.skip.weight", "b16.skip.resample_filter", "b8.resample_filter", "b8.conv0.weight", "b8.conv0.bias", "b8.conv0.resample_filter", "b8.conv1.weight", "b8.conv1.bias", "b8.conv1.resample_filter", "b8.skip.weight", "b8.skip.resample_filter", "b4.conv.weight", "b4.conv.bias", "b4.conv.resample_filter", "b4.fc.weight", "b4.fc.bias", "b4.out.weight", "b4.out.bias".

It seems something wrong with get_keys(ckpt, "discriminator"). However, get_keys(ckpt, "encoder") and get_keys(ckpt, "inverter") work well.

May I know the md5 of sfe.editor_light.pt? I want to check the model download completely or not.

retir commented 2 months ago

Hi, thanks for your report!

Regarding the first error, it looks like you are using Python 3.11, while we recommend using 3.10. Using other Python versions can cause this error. If the error remains after changing the python version, please give us information about your system and the versions of the packages installed in env.

Regarding the second error, it seems that you downloaded new weights but did not update the code. Previously our code tried to load discriminator weights from the encoder checkpoint, but the new version of the encoder weights (sfe_editor_light.pt) does not contain these weights. So we change the code a little bit, you can see it in models/methods.py lines 66-71 and line 77. You need to just update the code.

Please respond if this solves your problem so we can close the issue.

LiXm1002 commented 2 months ago

Oh, thanks, it works.