chrispiech / DeepKnowledgeTracing

source code for the paper Deep Knowledge Tracing
270 stars 89 forks source link

Is there an appoximate bias in AUC calculation #6

Open tswsxk opened 4 years ago

tswsxk commented 4 years ago

Notice the way AUC is calculated is by approximating to the true one where the code is

for i,p in ipairs(allPredictions) do
        if(p['truth'] == 1) then
            truePositives = truePositives + 1
        else
            falsePositives = falsePositives + 1
        end

        local guess = 0
        if(p['pred'] > 0.5) then guess = 1 end
        if(guess == p['truth']) then correct = correct + 1 end

        local fpr = falsePositives / totalNegatives
        local tpr = truePositives / totalPositives
        if(i % 500 == 0) then
            if lastFpr ~= nil then
                local trapezoid = (tpr + lastTpr) * (fpr - lastFpr) *.5
                auc = auc + trapezoid
            end 
            lastFpr = fpr
            lastTpr = tpr
        end
        if(recall == 1) then break end
    end

Wondering whether this appoximating calculation may affect the auc value.