search

Islanna / DynamicReLU

Implementation of Dynamic ReLU on Pytorch
204 stars 38 forks source link

About reduction ratios R #2

Open bobo0810 opened 4 years ago

bobo0810 commented 4 years ago

Thanks for your working! I have some questions:

  1. That reduction ratios R=8 is the best trade-off in the paper , but the code defaults to 4, so why?
  2. In the 800,000 person face classification task, what would you recommend the value of R? @Islanna Thank you again~
daixiangzi commented 4 years ago

Thanks for your working! I have some questions:

  1. That reduction ratios R=8 is the best trade-off in the paper , but the code defaults to 4, so why?
  2. In the 800,000 person face classification task, what would you recommend the value of R? @Islanna Thank you again~

hello,use dynamic relu in your task ,acc is ok?

bobo0810 commented 4 years ago

Thanks for your working! I have some questions:

  1. That reduction ratios R=8 is the best trade-off in the paper , but the code defaults to 4, so why?
  2. In the 800,000 person face classification task, what would you recommend the value of R? @Islanna Thank you again~

hello,use dynamic relu in your task ,acc is ok?

I haven't experimented...

Islanna commented 4 years ago

Hi!

The main reason is that I mainly used DyReLU to experiment with Speech-To-Text task. Sound's models and preprocessing are a little different from models for images. For example, often conv1d is used instead of conv2d.

So, the reduction ratio = 2-4 is optimal for sound. If you want to work with images maybe you should change it to 8 as in the paper.

bobo0810 commented 4 years ago

Hi!

The main reason is that I mainly used DyReLU to experiment with Speech-To-Text task. Sound's models and preprocessing are a little different from models for images. For example, often conv1d is used instead of conv2d.

So, the reduction ratio = 2-4 is optimal for sound. If you want to work with images maybe you should change it to 8 as in the paper.

Thanks reply！According to the paper, ratio = 8 is the best trade-off. Although ratiobn=4 works better, but it is very weak. Therefore, for better performance, choosing ration=4 is a suitable choice. Is it right？