Quantify the accuracy of the model output

[hilenamin]

Explore approaches to quantify the accuracy of the model output.

• [x] Define a a standard panel of users that will provide feedback
• [x] Define scoring method
• [x] Validate score with existing example output

[ymitiku]

Define a standard panel of users that will provide feedback Eight peoples are chosen to evaluate dataset available. Each user is given 12 videos of animation which is 30 seconds long each. This videos are recorded from training data currently available with confused and confident labels. Define scoring method The voting mechanism chosen is Majority Rule (hard) voting. The ground truth will be assigned to value decided by majority vote. Validate score with existing example output 32 Videos of 16 videos for each emotion(confident and confused) is recorded and distributed nearly in random manner for each person(12 videos per person). The following result is obtained from the feedback

• 29(~ 90.6%) videos are correctly labeled from which one is labeled by single person
• 2(~ 6.3%) videos are incorrectly labeled
• 1(~ 3.1% ) with equal number of votes for both confused and confident(2 voters)