search

akusok / hpelm

High performance implementation of Extreme Learning Machines (fast randomized neural networks).
Other
191 stars 61 forks source link

Confusion matrix and multiclass model #16

Open ilame opened 6 years ago

ilame commented 6 years ago

Hello,

Thanks

akusok commented 6 years ago

Hello,

Confusion matrix in ELM is not well implemented, I would suggest you using Scikit-Learn’s functionality instead: http://scikit-learn.org/stable/modules/model_evaluation.html#confusion-matrix http://scikit-learn.org/stable/modules/model_evaluation.html#confusion-matrix

Regarding classes, ELM/HPELM accept one-hot encoding, meaning that for 6 classes you should have 6 outputs where correct class has value +1 and all others have values 0. For example, class 2 is encoded as [0, 1, 0, 0, 0, 0].

Best regards, Anton Akusok

On 4 May 2018, at 13.26, ilame notifications@github.com wrote:

Hello,

I get this error message when using confusion() function: "Confusion matrix works only for regression" How can I compute confusion matrix for multiclass classification? -When defining the hpelm object like this: model= hpelm.ELM(inputs=50 , outputs=6, classification='mc') It gives me an error message saying: "T has wrong dimensionality: expected classes 6, found 1" Although I have 6 classes, when I change outputs to 1, it works fine and computes the model error. Any explanation please, I would be grateful! Thanks

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/akusok/hpelm/issues/16, or mute the thread https://github.com/notifications/unsubscribe-auth/ACgnerLTP-83mfB0jNVO4jVRFu6YinGdks5tvCy8gaJpZM4Tybc4.

ilame commented 6 years ago

Hello,

Thank you so much for your reply, it works fine now. However, for calculating the confusion matrix, I got some errors and I noticed that the predicted outputs are numeric (negative and positive) between 0 and 1 (like probabilities) and I found in the source code that for multi-label (ml) classification outputs are correct predictions if their value is greater than 0.5 My question is: should I consider the same logic for my multi-class (mc) problem ? if so, I must sort the predictions first before calculating the confusion matrix? is the best way?

Kind Regards Ilham

On 21 May 2018 at 09:48, Anton Akusok notifications@github.com wrote:

Hello,

Confusion matrix in ELM is not well implemented, I would suggest you using Scikit-Learn’s functionality instead: http://scikit-learn.org/stable/modules/model_evaluation.html#confusion- matrix http://scikit-learn.org/stable/modules/model_ evaluation.html#confusion-matrix

Regarding classes, ELM/HPELM accept one-hot encoding, meaning that for 6 classes you should have 6 outputs where correct class has value +1 and all others have values 0. For example, class 2 is encoded as [0, 1, 0, 0, 0, 0].

Best regards, Anton Akusok

On 4 May 2018, at 13.26, ilame notifications@github.com wrote:

Hello,

I get this error message when using confusion() function: "Confusion matrix works only for regression" How can I compute confusion matrix for multiclass classification? -When defining the hpelm object like this: model= hpelm.ELM(inputs=50 , outputs=6, classification='mc') It gives me an error message saying: "T has wrong dimensionality: expected classes 6, found 1" Although I have 6 classes, when I change outputs to 1, it works fine and computes the model error. Any explanation please, I would be grateful! Thanks

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub < https://github.com/akusok/hpelm/issues/16>, or mute the thread < https://github.com/notifications/unsubscribe-auth/ACgnerLTP- 83mfB0jNVO4jVRFu6YinGdks5tvCy8gaJpZM4Tybc4>.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/akusok/hpelm/issues/16#issuecomment-390607124, or mute the thread https://github.com/notifications/unsubscribe-auth/AJIDzg7pVzd6EbQ5vICCAtkTwUNZaPJpks5t0o1TgaJpZM4Tybc4 .

ilame commented 6 years ago

Hello,

I think It is solved by using : confusion_matrix(y_true, y_pred.argmax(axis=1)) ; y_true represents the true outputs before applying OneHotEncoder; y_pred= model.predict(X_test)

Kind Regards Ilham

On 21 May 2018 at 20:15, Ilham ilhammaj@gmail.com wrote:

Hello,

Thank you so much for your reply, it works fine now. However, for calculating the confusion matrix, I got some errors and I noticed that the predicted outputs are numeric (negative and positive) between 0 and 1 (like probabilities) and I found in the source code that for multi-label (ml) classification outputs are correct predictions if their value is greater than 0.5 My question is: should I consider the same logic for my multi-class (mc) problem ? if so, I must sort the predictions first before calculating the confusion matrix? is the best way?

Kind Regards Ilham

On 21 May 2018 at 09:48, Anton Akusok notifications@github.com wrote:

Hello,

Confusion matrix in ELM is not well implemented, I would suggest you using Scikit-Learn’s functionality instead: http://scikit-learn.org/stable/modules/model_evaluation. html#confusion-matrix http://scikit-learn.org/stabl e/modules/model_evaluation.html#confusion-matrix

Regarding classes, ELM/HPELM accept one-hot encoding, meaning that for 6 classes you should have 6 outputs where correct class has value +1 and all others have values 0. For example, class 2 is encoded as [0, 1, 0, 0, 0, 0].

Best regards, Anton Akusok

On 4 May 2018, at 13.26, ilame notifications@github.com wrote:

Hello,

I get this error message when using confusion() function: "Confusion matrix works only for regression" How can I compute confusion matrix for multiclass classification? -When defining the hpelm object like this: model= hpelm.ELM(inputs=50 , outputs=6, classification='mc') It gives me an error message saying: "T has wrong dimensionality: expected classes 6, found 1" Although I have 6 classes, when I change outputs to 1, it works fine and computes the model error. Any explanation please, I would be grateful! Thanks

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub < https://github.com/akusok/hpelm/issues/16>, or mute the thread < https://github.com/notifications/unsubscribe-auth/ ACgnerLTP-83mfB0jNVO4jVRFu6YinGdks5tvCy8gaJpZM4Tybc4>.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/akusok/hpelm/issues/16#issuecomment-390607124, or mute the thread https://github.com/notifications/unsubscribe-auth/AJIDzg7pVzd6EbQ5vICCAtkTwUNZaPJpks5t0o1TgaJpZM4Tybc4 .

akusok commented 6 years ago

Glad that it worked!

BR, Anton

On 22 May 2018, at 15.19, ilame notifications@github.com wrote:

Hello,

I think It is solved by using : confusion_matrix(y_true, y_pred.argmax(axis=1)) ; y_true represents the true outputs before applying OneHotEncoder; y_pred= model.predict(X_test)

Kind Regards Ilham

On 21 May 2018 at 20:15, Ilham <ilhammaj@gmail.com mailto:ilhammaj@gmail.com> wrote:

Hello,

Thank you so much for your reply, it works fine now. However, for calculating the confusion matrix, I got some errors and I noticed that the predicted outputs are numeric (negative and positive) between 0 and 1 (like probabilities) and I found in the source code that for multi-label (ml) classification outputs are correct predictions if their value is greater than 0.5 My question is: should I consider the same logic for my multi-class (mc) problem ? if so, I must sort the predictions first before calculating the confusion matrix? is the best way?

Kind Regards Ilham

On 21 May 2018 at 09:48, Anton Akusok <notifications@github.com mailto:notifications@github.com> wrote:

Hello,

Confusion matrix in ELM is not well implemented, I would suggest you using Scikit-Learn’s functionality instead: http://scikit-learn.org/stable/modules/model_evaluation http://scikit-learn.org/stable/modules/model_evaluation. html#confusion-matrix <http://scikit-learn.org/stabl http://scikit-learn.org/stabl e/modules/model_evaluation.html#confusion-matrix>

Regarding classes, ELM/HPELM accept one-hot encoding, meaning that for 6 classes you should have 6 outputs where correct class has value +1 and all others have values 0. For example, class 2 is encoded as [0, 1, 0, 0, 0, 0].

Best regards, Anton Akusok

On 4 May 2018, at 13.26, ilame <notifications@github.com mailto:notifications@github.com> wrote:

Hello,

I get this error message when using confusion() function: "Confusion matrix works only for regression" How can I compute confusion matrix for multiclass classification? -When defining the hpelm object like this: model= hpelm.ELM(inputs=50 , outputs=6, classification='mc') It gives me an error message saying: "T has wrong dimensionality: expected classes 6, found 1" Although I have 6 classes, when I change outputs to 1, it works fine and computes the model error. Any explanation please, I would be grateful! Thanks

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub < https://github.com/akusok/hpelm/issues/16 https://github.com/akusok/hpelm/issues/16>, or mute the thread < https://github.com/notifications/unsubscribe-auth/ https://github.com/notifications/unsubscribe-auth/ ACgnerLTP-83mfB0jNVO4jVRFu6YinGdks5tvCy8gaJpZM4Tybc4>.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <https://github.com/akusok/hpelm/issues/16#issuecomment-390607124 https://github.com/akusok/hpelm/issues/16#issuecomment-390607124>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AJIDzg7pVzd6EbQ5vICCAtkTwUNZaPJpks5t0o1TgaJpZM4Tybc4 https://github.com/notifications/unsubscribe-auth/AJIDzg7pVzd6EbQ5vICCAtkTwUNZaPJpks5t0o1TgaJpZM4Tybc4> .

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/akusok/hpelm/issues/16#issuecomment-390969553, or mute the thread https://github.com/notifications/unsubscribe-auth/ACgneonpIycoZbYpsTEoLXWuDHVW4rHwks5t1AJvgaJpZM4Tybc4.