some problems about updating style encoder network

[newExplore-hash]

train the generator

g_loss, g_losses_latent = compute_g_loss(nets, args, x_real, y_org, y_trg, z_trgs=[z_trg, z_trg2], masks=masks, face_mask=face_mask) self._reset_grad() g_loss.backward() optims.generator.step() optims.mapping_network.step() optims.style_encoder.step()

g_loss, g_losses_ref = compute_g_loss(nets, args, x_real, y_org, y_trg, x_refs=[x_ref, x_ref2], masks=masks, face_mask=face_mask) self._reset_grad() g_loss.backward() optims.generator.step()

Hi, I was very curious why the style encoder isn't updated when computing the loss with reference images.

[toculin38]

You can't update style encoder when training the discriminator.

[newExplore-hash]

You can't update style encoder when training the discriminator.

no, this is generator, you can find this code in solver.py(line122 - 134 )

[toculin38]

no, this is generator, you can find this code in solver.py(line122 - 134 )

Well, you quoted the discriminator part before you edited it.

To your question, my inference is probably for some consistency issue, style encoder will learn to output similar results for any image as input to minimize the loss "Style Reconstruction" term.

[yunjey]

@newExplore-hash Sorry for a late reply. We observed that the style reconstruction loss in latent-guided synthesis is enough to guide the style encoder to extract the styles of reference images. When we trained the style encoder with other losses, we found that the performance was highly degraded.