winkjs / wink-naive-bayes-text-classifier

Naive Bayes Text Classifier
http://winkjs.org/wink-naive-bayes-text-classifier/
MIT License
39 stars 9 forks source link

Can I do "Open Questions" with wink? #24

Closed thestonechat closed 3 years ago

thestonechat commented 3 years ago

Hey everyone!

So, I couldn't find any docs about this so I was making my own take on this, by making a history, which bot goes thru when question is asked and well thinks of answers to them too. but I got a problem. Wink uses negative numbers too! so im wondering if I can either: Use 0 to 1 numbers or if there is some way around it too!

Last time I asked you guys, you were REALLY helpful! Thanks for the work and stuff you make is amazing!

(also rephrasing what I mean by open questions: basically when asking bot lets say "who was nikola tesla", and then asking "did he love pigeons" I want it to know that I mean nikola tesla and do the processing then 😊)

rachnachakraborty commented 3 years ago

Hey @ignStonechat!

The open questions are possible with coreference resolution feature, which is currently not a part of winkjs library. It is definitely in our roadmap.

"Wink uses negative numbers too! so im wondering if I can either: Use 0 to 1 numbers or if there is some way around it too!"

We would be happy to help if you can let us know which negative numbers are you referring to?

Best, Rachna

thestonechat commented 3 years ago

Hey @ignStonechat!

The open questions are possible with coreference resolution feature, which is currently not a part of winkjs library. It is definitely in our roadmap.

"Wink uses negative numbers too! so im wondering if I can either: Use 0 to 1 numbers or if there is some way around it too!"

We would be happy to help if you can let us know which negative numbers are you referring to?

Best, Rachna

Hey! thanks for response. basically by negative numbers I mean on the bayes classifier when doing

classifier.computeOdds('some input')

responses can be like this image the input was obviously not correct (it was "what about his dog") so not relative to anything in the dataset. but when doing open questions that would be a MUST to calculate what gives the best answer! (I mean comparing the scores) I hope you get what I mean!

thestonechat commented 3 years ago

correction: the object is the response i'd get and the number by it is the score it got from "computeOdds"

rachnachakraborty commented 3 years ago

Hi @ignStonechat,

Thanks for clarifying.

Here is how the computation is taking place.

  1. Computing antilog of base 2 can give you the odds value;
  2. Probability = Odds/(1 + Odds)

In your case probability can be computed to get the values between 0 and 1

Hope this helps!

Best, Rachna

thestonechat commented 3 years ago

In your case probability can be computed to get the values between 0 and 1

How do I do that?! That would be amazing that's what I did with nlp.js but that library appeared to not be able to process big datasets that I needed to make my AI good. wink seems to handle a lot right now I loaded 2gb dataset yesterday and it went fine!

also please tell me if wink has similar limitations like max ammount of data it can handle!

thestonechat commented 3 years ago

(Im asking that because im kinda bad at math)

thestonechat commented 3 years ago

For that wouldn't I need to know MAX possible number (and I do not know that 😄)

sanjayaksaxena commented 3 years ago

Maximum amount of data is governed by Node.js max memory. If you get process out of memory with large datasets, following will help:

node --max_old_space_size=8192 index.js, where index.js is your code.