facebookresearch / StyleNeRF

This is the open source implementation of the ICLR2022 paper "StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis"
955 stars 91 forks source link

Difference between Generator in run_train and the one used to train pretrained checkpoints #29

Closed KyriaAnnwyn closed 2 years ago

KyriaAnnwyn commented 2 years ago

I get this error when loading pretrained network. The size of the layer was changed: Error loading: synthesis.fg_nerf.feat_out.weight torch.Size([64, 128, 1, 1]) torch.Size([256, 128, 1, 1])

Where can I modify the structure of generator to be the same as in pretrained checkpoint?

KyriaAnnwyn commented 2 years ago

Found it in stylenerf_ffhq.yaml: G_kwargs: class_name: "training.networks.Generator" z_dim: 512 w_dim: 512

mapping_kwargs:
    num_layers: ${spec.map}

synthesis_kwargs:
    # global settings
    num_fp16_res: ${num_fp16_res}
    channel_base: 1
    channel_max: 1024
    conv_clamp: 256
    kernel_size: 1
    architecture: skip
    upsample_mode: "nn_cat"

    z_dim_bg: 32
    z_dim: 0
    resolution_vol: 32
    resolution_start: 32
    rgb_out_dim: 256  <------------------------- this should be changed to 64

    use_noise: False
    module_name: "training.stylenerf.NeRFSynthesisNetwork"
    no_bbox: True
    margin: 0
    magnitude_ema_beta: 0.999
KyriaAnnwyn commented 2 years ago

Next problem:

Error loading: synthesis.b64.conv0.adapter.0.weight torch.Size([128, 512, 1, 1]) torch.Size([128, 1024, 1, 1])

KyriaAnnwyn commented 2 years ago

Found which param makes these changes: upsample_mode: "nn_cat" should be changed to "pixelshuffle"