search

jasp-stats / jasp-issues

This repository is solely meant for reporting of bugs, feature requests and other issues in JASP.
58 stars 29 forks source link

[Bug]: ROC for Naive Bayes Classification does not work #2794

Closed bayesfaqt closed 4 months ago

bayesfaqt commented 4 months ago

JASP Version

0.18.3

Commit ID

No response

JASP Module

Machine Learning

What analysis are you seeing the problem on?

ROC Plot for Naive Bayes Classification

What OS are you seeing the problem on?

Windows 11

Bug Description

Error - NBC-ROC

Data - CLF - NBC - Gender Weights.csv

Expected Behaviour

I would like to test Naive Bayes classification on simple data: a nominal-scale target and a nominal-scaled feature. I get statistics for the classification, data split details and a confusion matrix. However, I get the error - a screenshot of which is attached - when I check ROC in the Plots option

Steps to Reproduce

  1. Read data (file attached)
  2. Run ML/NB within Classification
  3. Drag Gender as Target
  4. Drag Weight>75 as Feature
  5. Check ROC curves under the Plots options ...

Log (if any)

JASP 2024-06-25 11_38_50 Engine 0.log JASP 2024-06-25 11_38_50 Desktop.log

Final Checklist

koenderks commented 4 months ago

The first thing I would advise you to do is to change the variable name 'weight > 75' to something that has only letters and no spaces. Something like 'highWeight' or something. I think the '>' could cause a problem. Let me know if that works!

bayesfaqt commented 4 months ago

Hi! Thanks for your instant reply. Changing the features name to Weight, get the same problem

image

tomtomme commented 4 months ago

confirmed on flatpak 0.18.3 and 0.19beta

koenderks commented 4 months ago

Okay, thanks for trying. I is indeed a bug, it will be fixed in the next release. In the meantime, here is a screenshot of the ROC-curves for your data if you use Set seed: 1 in the training parameters section. Setting the seed ensures that we use the same training and testset. You should be able to replicate the numbers in the table using this seed, and then the ROC-curves match.

image

If you need any more screenshots, don't hesitate to ask.

bayesfaqt commented 4 months ago

Thank you again for your prompt response

On Tue, 25 Jun, 2024, 12:25 Koen Derks, @.***> wrote:

Okay, thanks for trying. I is indeed a bug, it will be fixed in the next release. In the meantime, here is a screenshot of the ROC-curves for your data if you use 'Set seed: 1' in the training parameters section. Setting the seed ensures that we use the same training and testset. You should be able to replicate the numbers in the table using this seed, and then the ROC-curves match. image.png (view on web) https://github.com/jasp-stats/jasp-issues/assets/25059399/321534dc-0032-4026-9066-ffc5687f3ba3

If you need any more screenshots, don't hesitate to ask.

— Reply to this email directly, view it on GitHub https://github.com/jasp-stats/jasp-issues/issues/2794#issuecomment-2188122075, or unsubscribe https://github.com/notifications/unsubscribe-auth/BJNWV4EGBLYGVI743BNRO6LZJEH57AVCNFSM6AAAAABJ3EHSBKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBYGEZDEMBXGU . You are receiving this because you authored the thread.Message ID: @.***>

bayesfaqt commented 4 months ago

Great!

On Tue, 25 Jun, 2024, 13:13 bruno boutin, @.***> wrote:

Closed #2794 https://github.com/jasp-stats/jasp-issues/issues/2794 as completed via @.*** https://github.com/jasp-stats/jaspMachineLearning/commit/b5b6d7bc3cd4f344baf30f33b389ee63624890a4 .

— Reply to this email directly, view it on GitHub https://github.com/jasp-stats/jasp-issues/issues/2794#event-13276734415, or unsubscribe https://github.com/notifications/unsubscribe-auth/BJNWV4AHPKFIMQQP6BA5ZP3ZJENTFAVCNFSM6AAAAABJ3EHSBKVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMJTGI3TMNZTGQ2DCNI . You are receiving this because you authored the thread.Message ID: @.***>