search

mfbalin / Concrete-Autoencoders

112 stars 31 forks source link

Repeated features? #3

Open xitefanjisi opened 4 years ago

xitefanjisi commented 4 years ago

Thank you for your great feature selection method. However, I found that your method might lead to such a situation that the same features could be selected for multiple times. For example, after training, the method get_support() might output a result like [3, 4, 6, 2, 2, 5, 1, 0]. In this case, the second feature has been selected twice. Is it possible to avoid such cases using concrete autoencoders? Thank you!

a8-mlevy commented 4 years ago

I am noticing the same thing

Alexanderbertrand commented 3 years ago

We had a similar problem when using this method for selection of EEG channels. We were able to reduce the number of duplicate selections by adding a regularization term to the loss, which penalizes such duplicates. Paper here: https://homes.esat.kuleuven.be/~abertran/reports/TS_JNE_2021.pdf Github: https://github.com/AlexanderBertrandLab/Gumbel-Channel-Selection

mfbalin commented 2 years ago

In my experience, replicated features are caused by not having enough number of epochs to converge. When I was running experiments for the ICML paper, when I set the hyperparameters right, I never observed replicated features selected.