search

Cyanogenoid / dspn

[NeurIPS 2019] Deep Set Prediction Networks
https://arxiv.org/abs/1906.06565
MIT License
100 stars 18 forks source link

Question about the learning rate η #6

Open TTTREE opened 4 years ago

TTTREE commented 4 years ago

Hi! This is a nice work! I am studying your paper and trying to running the DSPN method on my own dataset. However, I find the repr_loss and set_loss can not convergence after some epoches. So I wonder if the learning rate η = 800 is suit to my dataset, which is CrowdHuman dataset. Or can you tell me how you adjust the learning rate in your experiments. Or could you please give me any advice about using DSPN on custom dataset? Thanks a lot!

Cyanogenoid commented 4 years ago

The way I set η is by looking what happens when the weights are randomly initialised, i.e. at the start of training. I started with a low η and increased it until I saw in the Tensorboard plots that with random init, the set elements were visible changing when doing the inner optimisation loop. For my datasets, I normalised all the set elements to be either in the range [0, 1] or [-1, 1]. Let me know if that helps.