search

NUS-HPC-AI-Lab / Neural-Network-Parameter-Diffusion

We introduce a novel approach for parameter generation, named neural network parameter diffusion (p-diff), which employs a standard latent diffusion model to synthesize a new set of parameters
787 stars 38 forks source link

Question about motivation #4

Closed ChongQingNoSubway closed 4 months ago

ChongQingNoSubway commented 4 months ago

Hello!! Thank you for revealing a good project.

Is the diffusion phenomenon limited only to SGD? Will the code related to your Figure 1 be released?

Thanks

1zeryu commented 4 months ago

Thanks for your feedback. We will try other optimizer, e.g, Adam. Meanwhile, you can change the config file in configs/task dir. For example, the optimizer in config/task/cifar100.yaml optimizer: _target_: torch.optim.SGD lr: 0.1 momentum: 0.9 weight_decay: 0.0005 can change to (Adam optimizer) optimizer: _target_: torch.optim.Adam lr: 0.1 weight_decay: 0.0005. We will run related experiments and are more than willing to share our results! If you have any question, feel free to ask us.

ChongQingNoSubway commented 4 months ago

Thanks