search

JingweiToo / Wrapper-Feature-Selection-Toolbox-Python

This toolbox offers 13 wrapper feature selection methods (PSO, GA, GWO, HHO, BA, WOA, and etc.) with examples. It is simple and easy to implement.
BSD 3-Clause "New" or "Revised" License
249 stars 69 forks source link

Question? #2

Closed Njzjhd closed 3 years ago

Njzjhd commented 3 years ago

Hi JingweiTOO,

Thank you for your hard works on Python version of the wrapper feature selection. For feature selection, shouldn't cross-validation be added when the fitness function is calculated. How should the fitness function change if for regression problems? I would like to give some hints or code.

JingweiToo commented 3 years ago

Hi,

The example is using holdout validation. If you want to use k-fold cross-validation, then you may change the code in fitness function. Also, you may change the fitness function for regression problem such as replace the KNN with logistic regression but also need to change the error to mean square error.

Njzjhd commented 3 years ago

Hi,

Thank you for the tip and the response. If, instead of using cross-validation, is there an accuracy defect problem?

------------------ 原始邮件 ------------------ 发件人: "JingweiToo/Wrapper-Feature-Selection-Toolbox-Python" <notifications@github.com>; 发送时间: 2021年1月22日(星期五) 上午10:50 收件人: "JingweiToo/Wrapper-Feature-Selection-Toolbox-Python"<Wrapper-Feature-Selection-Toolbox-Python@noreply.github.com>; 抄送: "至爱❤️"<2318109878@qq.com>;"Author"<author@noreply.github.com>; 主题: Re: [JingweiToo/Wrapper-Feature-Selection-Toolbox-Python] Question? (#2)

Hi,

The example is using holdout validation. If you want to use k-fold cross-validation, then you may change the code in fitness function. Also, you may change the fitness function for regression problem such as replace the KNN with logistic regression but also need to change the error to mean square error.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

hanamthang commented 3 years ago

Hi,

The example is using holdout validation. If you want to use k-fold cross-validation, then you may change the code in fitness function. Also, you may change the fitness function for regression problem such as replace the KNN with logistic regression but also need to change the error to mean square error.

The Logistic Regression you mentioned is only for binary problem, right?

Since error appear when I attempt to apply to continuous variables.

`in check_classification_targets(y) 167 if y_type not in ['binary', 'multiclass', 'multiclass-multioutput', 168 'multilabel-indicator', 'multilabel-sequences']: --> 169 raise ValueError("Unknown label type: %r" % y_type) 170 171

ValueError: Unknown label type: 'continuous'`

Just want to confirm that your codes are not for continuous variable at the moment.

JingweiToo commented 3 years ago

Hi,

My bad, the logistic regression is applied for binary classification but not to the regression. You may refer to other regression algorithms such as support vector regression, decision tree regression and linear regression.

kangwenhao commented 3 years ago

How to select the feature of regression model with this toolbox

JingweiToo commented 3 years ago

Fro the regression, you need to change the fitness function and replace the classifier with a regression model. Also, replace the error rate with a mean square error

Best Regards, Jingwei Too

On Wed, Mar 10, 2021 at 3:44 PM kangwenhao notifications@github.com wrote:

How to select the feature of regression model with this toolbox

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/JingweiToo/Wrapper-Feature-Selection-Toolbox-Python/issues/2#issuecomment-795033716, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHKG2G4NI2A7NNWUIEU57U3TC4PNTANCNFSM4WNZVHRA .

Njzjhd commented 3 years ago

I understand. Thank you again and have a great job!

------------------ 原始邮件 ------------------ 发件人: "JingweiToo/Wrapper-Feature-Selection-Toolbox-Python" @.>; 发送时间: 2021年3月10日(星期三) 下午3:46 @.>; @.**@.>; 主题: Re: [JingweiToo/Wrapper-Feature-Selection-Toolbox-Python] Question? (#2)

Fro the regression, you need to change the fitness function and replace the classifier with a regression model. Also, replace the error rate with a mean square error

Best Regards, Jingwei Too

On Wed, Mar 10, 2021 at 3:44 PM kangwenhao @.***> wrote:

> How to select the feature of regression model with this toolbox > > — > You are receiving this because you commented. > Reply to this email directly, view it on GitHub > <https://github.com/JingweiToo/Wrapper-Feature-Selection-Toolbox-Python/issues/2#issuecomment-795033716&gt;, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AHKG2G4NI2A7NNWUIEU57U3TC4PNTANCNFSM4WNZVHRA&gt; > . >

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

kangwenhao commented 3 years ago

Can you be specific? I am changing pso.py Or demo_ PSO.py

Njzjhd commented 3 years ago

PSO is a classical optimization algorithm. It seems that it would be very interesting to combine the ideas of PSO by fusing other algorithms

------------------ 原始邮件 ------------------ 发件人: "JingweiToo/Wrapper-Feature-Selection-Toolbox-Python" @.>; 发送时间: 2021年3月10日(星期三) 下午3:53 @.>; @.**@.>; 主题: Re: [JingweiToo/Wrapper-Feature-Selection-Toolbox-Python] Question? (#2)

Can you be specific? I am changing pso.py Or demo_ PSO.py

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

Njzjhd commented 3 years ago

For now, I got your inspiration and I think I can solve this work. pso is a classical optimization algorithm. It seems that it would be very interesting to combine the ideas of PSO by fusing other algorithms

------------------ 原始邮件 ------------------ 发件人: "JingweiToo/Wrapper-Feature-Selection-Toolbox-Python" @.>; 发送时间: 2021年3月10日(星期三) 下午3:53 @.>; @.**@.>; 主题: Re: [JingweiToo/Wrapper-Feature-Selection-Toolbox-Python] Question? (#2)

Can you be specific? I am changing pso.py Or demo_ PSO.py

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

kangwenhao commented 3 years ago

具体怎么改适应度函数了？在回归里那个 c是什么啊？

kangwenhao commented 3 years ago

进行回归时，您需要更改适应度函数并将分类器替换为回归模型。另外，将误差率替换为均方误差。 … On Wed, Mar 10, 2021 at 3:44 PM kangwenhao @.***> wrote: How to select the feature of regression model with this toolbox — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHKG2G4NI2A7NNWUIEU57U3TC4PNTANCNFSM4WNZVHRA . 具体怎么改适应度函数了？在回归里那个c是什么啊？

kangwenhao commented 3 years ago

进行回归时，您需要更改适应度函数并将分类器替换为回归模型。另外，将误差率替换为均方误差。 … On Wed, Mar 10, 2021 at 3:44 PM kangwenhao @.***> wrote: How to select the feature of regression model with this toolbox — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHKG2G4NI2A7NNWUIEU57U3TC4PNTANCNFSM4WNZVHRA .

How to change the fitness function? What is that in the return?

Njzjhd commented 3 years ago

I think I see where you're going with this. But for the implementation of the algorithm, currently I mainly use python. matlab is too difficult for me. ------------------ 原始邮件 ------------------ 发件人: "JingweiToo/Wrapper-Feature-Selection-Toolbox-Python" @.>; 发送时间: 2021年3月10日(星期三) 下午4:04 @.>; @.**@.>; 主题: Re: [JingweiToo/Wrapper-Feature-Selection-Toolbox-Python] Question? (#2)

进行回归时，您需要更改适应度函数并将分类器替换为回归模型。另外，将误差率替换为均方误差。 … On Wed, Mar 10, 2021 at 3:44 PM kangwenhao @.***> wrote: How to select the feature of regression model with this toolbox — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHKG2G4NI2A7NNWUIEU57U3TC4PNTANCNFSM4WNZVHRA .

How to change the fitness function? What is that in the return?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

kangwenhao commented 3 years ago

我想我知道您要怎么做。但是对于算法的实现，目前我主要使用python。Matlab对我来说太难了。------------------＆nbsp;原始邮件＆nbsp; ------------------发件人：“ JingweiToo / Wrapper -Feature-Selection-Toolbox-Python“ @.> ;; 发送时间：＆nbsp; 2021年3月10日（星期三）下午4:04 @.> ;; @ 。** @ 。＆gt ;; 主题：Re：[JingweiToo / Wrapper-Feature-Selection-Toolbox-Python]有问题吗？（＃2…在2021年3月10日，星期三，3：44 PM kangwenhao @。*** ＆gt; 写道：如何使用此工具箱选择回归模型的功能-之所以收到此评论，是因为您发表了评论。直接回复此电子邮件，在GitHub < ＃2（评论）＆gt;上查看，或退订https://github.com/notifications/unsubscribe-auth/AHKG2G4NI2A7NNWUIEU57U3TC4PNTANCNFSM4WNZVHRA。如何更改健身功能？回报是什么？—您收到此消息是因为您创建了线程。直接回复此电子邮件，在GitHub上查看或取消订阅。

I use python, too。I need to do feature selection of regression model。But I am a cainiao, I will not change.

Njzjhd commented 3 years ago

This is a somewhat complicated process that I am exploring. We need to read more literature and related procedures ------------------ 原始邮件 ------------------ 发件人: "JingweiToo/Wrapper-Feature-Selection-Toolbox-Python" @.>; 发送时间: 2021年3月10日(星期三) 下午4:09 @.>; @.**@.>; 主题: Re: [JingweiToo/Wrapper-Feature-Selection-Toolbox-Python] Question? (#2)

我想我知道您要怎么做。但是对于算法的实现，目前我主要使用python。Matlab对我来说太难了。------------------＆nbsp;原始邮件＆nbsp; ------------------发件人：“ JingweiToo / Wrapper -Feature-Selection-Toolbox-Python“ @.> ;; 发送时间：＆nbsp; 2021年3月10日（星期三）下午4:04 @.> ;; @ 。 @ 。＆gt ;; 主题：Re：[JingweiToo / Wrapper-Feature-Selection-Toolbox-Python]有问题吗？（＃2…在2021年3月10日，星期三，3：44 PM kangwenhao @。*** ＆gt; 写道：如何使用此工具箱选择回归模型的功能-之所以收到此评论，是因为您发表了评论。直接回复此电子邮件，在GitHub < ＃2（评论）＆gt;上查看，或退订https://github.com/notifications/unsubscribe-auth/AHKG2G4NI2A7NNWUIEU57U3TC4PNTANCNFSM4WNZVHRA。如何更改健身功能？回报是什么？—您收到此消息是因为您创建了线程。直接回复此电子邮件，在GitHub上查看或取消订阅。

I use python, too。I need to do feature selection of regression model。But I am a cainiao, I will not change.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

Njzjhd commented 3 years ago

<A wrapper approach-based key temperature point selection and thermal error modeling method>。Maybe it will help you, you can read it

------------------ 原始邮件 ------------------ 发件人: "JingweiToo/Wrapper-Feature-Selection-Toolbox-Python" @.>; 发送时间: 2021年3月10日(星期三) 下午4:03 @.>; @.**@.>; 主题: Re: [JingweiToo/Wrapper-Feature-Selection-Toolbox-Python] Question? (#2)

进行回归时，您需要更改适应度函数并将分类器替换为回归模型。另外，将误差率替换为均方误差。 … On Wed, Mar 10, 2021 at 3:44 PM kangwenhao @.***> wrote: How to select the feature of regression model with this toolbox — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHKG2G4NI2A7NNWUIEU57U3TC4PNTANCNFSM4WNZVHRA . 具体怎么改适应度函数了？在回归里那个c是什么啊？

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

fatima24-a commented 3 years ago

Hi, How can we check which features are selected because code just show selected features number. If we want to show which features are selected then how to show it?