why set 1e-7 norm eps - Githubissues

mit-han-lab / efficientvit

EfficientViT is a new family of vision models for efficient high-resolution vision.

Apache License 2.0

1.6k stars 142 forks source link

why set 1e-7 norm eps #51

Open seefun opened 7 months ago

seefun commented 7 months ago

1e-7 is a weird norm eps, odd choice. Is there any purpose to do this？ And I noticed that the sam model and the cls model have different eps settings.