how to choose optimal random direction?

tomgoldstein / loss-landscape

Code for visualizing the loss landscape of neural nets

MIT License

2.79k stars 396 forks source link

how to choose optimal random direction? #31

Open weiHelloWorld opened 3 years ago

weiHelloWorld commented 3 years ago

Hi, thanks for sharing this code!

I am recently trying with this method, and noticed that the direction we select has a significant impact on the loss curve. I am just wondering if anyone has seen similar behavior, and how we should select an "optimal" direction for analysis?

Thanks!