How to setup feedback related to a specific bot?

[jdd-software]

I would like to setup these possibile feedback on Hydrant (HD).

• fn = False negative (reported stuff)
• tp = True positive
• fp = False positive
• nc = Non constructive (No-longer needed flag)
• sk = Skip comment it's hard to classify

Is it enough that I send'em once via Bot/BotRegisterFeedbackTypesPost, how do I select icon. Maybe a bot setup web page would be better?

[Filnor]

Shouldn't tn be fn (false negative), as the bot didn't catch it (but it should have)?

[jdd-software]

@pbdevch That's an old discussion https://chat.stackoverflow.com/transcript/111347?m=34014773#34014773... or old Fish decided that true/false=is relative to if it should be detected, positive/negative is related to if it was detected.... who am I to tell that his is wrong :). Anyway Natty uses tn so lets not get it even more confusing.

[double-fault]

@jdd-software Natty is wrong too. I believe that it'll be best if correct feedbacks are introduced into new bots, instead of introducing wrong ones because Natty got it wrong.

[jdd-software]

@Fortunate-MAN so you are saying that Tuna is wrong O_O, That needs to be a true negative.... I'm not sure it's wrong, read Tuna's chat message it kind of make sense to me. Depends how you interpret the words..

[rjrudman]

FWIW, I agree with using the correct terminology here. Natty is known to be wrong, but it's fixable. I'd rather start out correctly, rather than having to go and change it retroactively.

In any case, for the actual issue; I'm planning on adding a dashboard to configure feedback for bots. In the meantime, yes, you'll need to use Bot/BotRegisterFeedbackTypesPost. The 'Icon' field will accept a unicode character as the icon. For example, a tick or a cross

[graemeberry]

True negative sounds right to me. It's being manually reported so someone thinks this is "true" (like "true positive"). It wasn't reported automatically so it's "negative" hence "true negative".

[jdd-software]

Let's push in some machine learning definitions and decide which is correct

https://developers.google.com/machine-learning/crash-course/classification/true-false-positive-negative

A true positive is an outcome where the model correctly predicts the positive class. Similarly, a true negative is an outcome where the model correctly predicts the negative class.

A false positive is an outcome where the model incorrectly predicts the positive class. And a false negative is an outcome where the model incorrectly predicts the negative class.

hmm to me it's seems like the model incorrectly predicts negative class (it think the answer/comment is ok, but it is not).

Which would support the idea that we should call it false negative, I have updated the feedback guide, Thanks all for your input!

[Bhargav-Rao]

No, the correct term is false negative.

Tp, Fp, Tn and Fn are all a part of a statistical measures for classifiers. They are used to measure specificity and sensitivity of a a particular classifier.

Here the classifier is Heat Detector. In the following image let red be the rude comments and green be the proper comments. The dots on the right side are the ones which Heat Detector classified and the ones on the left are the ones which it didn't

When a classifier classifies any point, it is considered positive. When a classifier does not classify a point, it is considered negative.

So now, the right hand side is considered as positive and the left hand side as negative.

When the prediction is correct, it is classified as True. When the prediction is incorrect, it is classified as False.

A manual report:

• is not classified by HD : hence it is negative
• is incorrectly predicted as "not heat": hence it is false

Therefore manual reports are false negatives

You can consider this table for a nice explanation:

		Actual Label
		True .	False
Classified as	True	tp	fp
	False	fn	tn

[jdd-software]

@rjrudman returning to original question :) sending from bot I would do something like this

FeedbackType fb1 = new FeedbackType();
fb1.setName(FeedBack.TP.toString()); //tp
fb1.setColour("green");
fb1.setIcon("✓");
fb1.setIsActionable(true);
fb1.setRequiredActions(0);

1. In color you do I need to send "green", "red" etc or is also hex values ok like #F35632?
2. Icon example which is the best format?
3. isActionable??, what is this what does it do?
4. setRequieredActions??, , what is this what does it do?

This is current result http://higgs.sobotics.org:5555/report/186, if you are logged in you can probably see my setup, my html encoding is not working :)

.

[Filnor]

3. Sounds to me like a setting if we can give feedback from higgs or not (just my assumptions/thoughts)

[rjrudman]

I've removed the API endpoint for bots to manage setting up feedback. As a replacement, we now have a web view to manage them, which will be only visible to admins and bot owners. Here's yours: http://higgs.sobotics.org:5555/admin/bot/feedbackTypes/1

I badly implemented the bot endpoint, and caused https://github.com/SOBotics/Higgs/issues/29. If you still want to programatically setup the feedback, see admin/ViewBotFeedbackTypes and admin/EditBotFeedbackTypes.

SetRequiredActions has been removed. It originally was supposed to restrict feedback to users with a particular role. May implement this in the future, but doesn't look like it'll be useful.

IsActionable doesn't do anything yet. The idea behind it, is that we want Higgs to be able to figure out which reports have been 'acted on', so we can build a review queue. Setting IsActionable to false excludes the feedback from the calculation.

Icon is just a string, but it's designed to take a unicode character. See the above link for an example.

Colour takes any browser supported colour. green or #ff0000 both work