PSO optimization for binary classification

[jkseven-art]

Hi @thieu1995, I was looking for a metaheuristic algorithm to optimize neural network's weights. I've just found mealpy here and trying to use that. I've a question about PSO, SSA or any kind of swarm based algorithm. can we use them in binary classification problem? or we have to modify your code in order to get what we are looking for? I watched your tutorial videos that u used GWO for a classification dataset. but what about particle swarm optimization? we should make it a binary optimization or not? Regards; Jacob.

[thieu1995]

@jacobkndy ,

Everything I said and did in the video is important. Please don't skip any part of it. Watch this video: https://www.youtube.com/watch?v=auq7Na1Meus
at 59:40 - Using FPA optimizer instead of GWO optimizer (I even noted with the timestamp in video description) You can use any algorithm in this library (PSO, WOA, SSA, whatever...) to solve this kind of problem. That is the purpose of this library. For other libraries, you have to design your own operators for each problem. But we don't do that here. We just need to create an object of optimizer you want to use and call the function solve().

For the question about binary classification. I don't know, you have to try every algorithm you want to and compare the results. Maybe PSO gives better results than GWO. It also depends on your problem (your dataset)...

Just remember, no algorithm is better than all other algorithms in all problems (No-Free-Launch Theory). Or there will be no best algorithm in all problems.

[jkseven-art]

@thieu1995 You're right sir. this library suits to all kind of problems but I read somewhere that for using PSO algorithm in binary classification, we have to make it binary optimizer because it's a kind of non-binary algorithm. my question was that should I edit some part of code in order to get a binary PSO for this kind of problems or not? or maybe u think having "sigmoid" function as activation is enough? Thank you for being there.

[thieu1995]

@jacobkndy I got you now. At first, I thought you are talking about the binary classification problem. But you are actually talking about binary PSO. But still, you are misunderstanding between the problem and the optimizer. As I said, they are not related to each other. Binary classification is a problem, if you use a neural network then its output is the probability of two classes (0 or 1). To solve this problem with a neural network, they usually use a gradient-based optimizer like (adam, adelta,...) (continuous domain and continuous problem).

Binary PSO is just a way to represent a solution in form of a binary vector like [0, 0, 1]. For normal PSO, it is usually a real value vector like [0.1, -0.2, 1.5,...]. Binary used in Metaheuristic Algorithm is a term originally come from Genetic Algorithm. Because a Chromosome of a human's DNA sequence is represented by [g1, g2,..., gn] in which gi is a gene and coded by m bits (binary representation). So they use this kind of representation for binary Genetic Algorithm only. But for the continuous domain, they use a real value encoded mechanism like above. https://prateek-mishra.medium.com/binary-genetic-algorithm-19936e755271 Other algorithms with the word Binary in front of them just like copy-paste the form of GA. And it is actually not good for their algorithm because they design their algorithm by the continuous operator (real value). But use binary representation to encode a solution.

So I suggest you don't do that. Just understand your problem is a discrete problem or the continuous problem. And select algorithms you want to use. For example in my tutorial videos:

• Mealpy replaces Gradient Descent of Neural Network is the continuous problem.
• Mealpy optimizes hyper-parameter of Neural Network is mixed of discrete variables and continuous variables.

So forget about binary PSO, just use the normal algorithm for the continuous problem since this library is for the continuous problem. But It can also solve the discrete problem as I did with the example of hyper-parameter tuning.

I just google it.

[jkseven-art]

@thieu1995 I appreciate you. I take ur time a lot. actually I saw you in the video that using pima-indians-diabetes (which is a kind of binary classification) and thought that my dataset is like this. for example I have 12 feature (input) and one output which could be [0 or 1]. I know that my problem is a kind of discrete problem and for this reason I wanted to learn how to make binary PSO, but when I saw these videos and because my dataset was like "pima-indians-diabetes" so I guess maybe I can use PSO algorithm. that's the story. thanks for your time and your attention.

[thieu1995]

@jacobkndy ,

As you said above, "I was looking for a metaheuristic algorithm to optimize neural network's weights". Your problem is continuous, not discrete. Your variables (solutions) are continuous values from -1 to 1 (weights of network usually set from -1 to 1).

If you want to use metaheuristics to find the best hyper-parameter of the classification model. Then your problem can be a continuous problem, discrete problem, or mix between them, it depends on the parameter you choose is continuous or discrete.

I think I will close this issue now. I explained everything you need to know. May need to re-read some more times to get what did I say.

[jkseven-art]

sir, you did too much to me. Your videos and messages was so helpful. I appreciate you. Thank you again. J.

On Mon, Dec 20, 2021 at 5:51 PM Nguyen Van Thieu @.***> wrote:

@jacobkndy https://github.com/jacobkndy ,

As you said above, "I was looking for a metaheuristic algorithm to optimize neural network's weights". Your problem is continuous, not discrete. Your variables (solutions) are continuous values from -1 to 1 (weights of network usually set from -1 to 1).

If you want to use metaheuristics to find the best hyper-parameter of the classification model. Then your problem can be a continuous problem, discrete problem, or mix between them, it depends on the parameter you choose is continuous or discrete.

I think I will close this issue now. I explained everything you need to know. May need to re-read some more times to get what did I say.

— Reply to this email directly, view it on GitHub https://github.com/thieu1995/mealpy/issues/37#issuecomment-997967307, or unsubscribe https://github.com/notifications/unsubscribe-auth/AW7A7VHVPRCJ26JOFYYZPTTUR43VLANCNFSM5KKVNA5A . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>