search

memphis-iis / mofacts

8 stars 1 forks source link

Saved probabilities for each item are incorrect #1343

Closed imrryr closed 11 months ago

imrryr commented 11 months ago

p.clusterPrevious CalculatedProbabilities seems to be storing value after the trial are completed, rather than before. So for example, they are extremely high, near 1, even though they are selected around .7

imrryr commented 11 months ago

There is still a fundamental misundertanding of the spec here. It is supposed to record this only when the trial is practiced, so since this item was not practiced, these PreviousCalculatedProbabilitie lists should be empty, but they actually have 31 values...

PreviousCalculatedProbabilities, both for stim or cluster, only records the values that resulted in each drill, so only if the overallClusterHistory was a vector of 31 1s ans 0s could the clusterPreviousCalculatedProbabilities be a list of 31 probabilities.

In fact clusterPreviousCalculatedProbabilities should be the probability that resulted in each drill trial being selected...

So,

[1, 0] clusterPreviousCalculatedProbabilities indicates 1 correct then 1 wrong [.65,.75] indicates the correct response was given after probability was computed at .65 prior to selection, then the incorrect response was given when the model expected a 75% chance of recall.

it works the same for stimPreviousCalculatedProbabilities

nswerText : "halberd" clusterIndex : 12 clusterOutcomeHistory : [] clusterPreviousCalculatedProbabilities : (31) [0.009877310559999294, 0.009877310559999294, 0.009877310559999294, 0.009877310559999294, 0.014778765102238363, 0.01873228170042944, 0.01873228170042944, 0.022186964070316003, 0.022186964070316003, 0.025321773242573956, 0.025321773242573956, 0.028230134679139603, 0.028230134679139603, 0.028230134679139603, 0.03096825783173243, 0.03357312501955103, 0.03607062541619069, 0.03607062541619069, 0.03607062541619069, 0.03607062541619069, 0.038479730721785, 0.04081484170249863, 0.04308719977170954, 0.04308719977170954, 0.04308719977170954, 0.04308719977170954, 0.04530578262753169, 0.028439278439125932, 0.02198272171316279, 0.0231073746389156, 0.02421212587302715] hintLevel : 0 i : 9 instructionQuestionResult : null overallOutcomeHistory : (17) [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 1, 1] pFunc : {} questionFailureCount : 0 questionSecsPracticingOthers : 0 questionSecsSinceFirstShown : 0 questionSecsSinceLastShown : 0 questionStudyTrialCount : 0 questionSuccessCount : 0 questionTotalTests : 0 resp : {KCId: 145, hintLevel: null, priorCorrect: 0, allTimeCorrect: 0, allTimeIncorrect: 0, …} responseFailureCount : 0 responseOutcomeHistory : [] responseSecsSinceLastShown : 0 responseStudyTrialCount : 0 responseSuccessCount : 0 stim : {stimuliSetId: 87, stimulusFileName: 'Radicalstimsv3.json', stimulusKC: 870055, clusterKC: 870055, responseKC: 145, …} stimFailureCount : 0 stimIndex : 0 stimOutcomeHistory : [] stimParameters : (2) [-4.6075886, 0.7] stimPreviousCalculatedProbabilities : (31) [0.009877310559999294, 0.009877310559999294, 0.009877310559999294, 0.009877310559999294, 0.014778765102238363, 0.01873228170042944, 0.01873228170042944, 0.022186964070316003, 0.022186964070316003, 0.025321773242573956, 0.025321773242573956, 0.028230134679139603, 0.028230134679139603, 0.028230134679139603, 0.03096825783173243, 0.03357312501955103, 0.03607062541619069, 0.03607062541619069, 0.03607062541619069, 0.03607062541619069, 0.038479730721785, 0.04081484170249863, 0.04308719977170954, 0.04308719977170954, 0.04308719977170954, 0.04308719977170954, 0.04530578262753169, 0.028439278439125932, 0.02198272171316279, 0.0231073746389156, 0.02421212587302715] stimResponseText : "halberd" stimSecsPracticingOthers : 0 stimSecsSinceFirstShown : 0 stimSecsSinceLastShown : 0 stimStudyTrialCount : 0 stimSuccessCount : 0 userCorrectResponses : 14 userTotalResponses : 16