How to set parameter list to multiple optimizers?

[kitaev-chen]

People usually use multiple module & optimizer on GAN model, for example:

moduleA = Generator()
moduleB = Discriminator()
moduleC = Predictor()

so the corresponding optimizers are:

optG = optim.Adam(Generator.parameters(), ...)
optD = optim.Adam(Discriminator.parameters(), ...)
optP = optim.Adam(Predictor.parameters(), ...)

For single module, the example show:

model = Model()
optimizer = optim.Adam([
                {'params': model.parameters()},
                {'params': awl.parameters(), 'weight_decay': 0} 
            ])

For the multiple modules above, how to set the parameters in optimizers? I can guess two options but they might be wrong: option1:

optG = optim.Adam(list(Generator.parameters()), ...)
optD = optim.Adam(list(Discriminator.parameters()), ...)
optP = optim.Adam(list(Predictor.parameters())+list(awl.parameters()), ...)

option2:

optG = optim.Adam(list(Generator.parameters())+list(awl.parameters()), ...)
optD = optim.Adam(list(Discriminator.parameters())+list(awl.parameters()), ...)
optP = optim.Adam(list(Predictor.parameters())+list(awl.parameters()), ...)

@Mikoto10032 Which one is correct?

Thanks!

[long123524]

Same question. Have you solved it yet? Thank you

[kitaev-chen]

Same question. Have you solved it yet? Thank you

No, I'm still thinking about it.