After going through the two modellings of the SAT problem in chapter 12 (in version 1.0.9 of the book), I was wondering if it was possible to do a further step forward and include the penalties in the problem.
Here is what I did (full version is here, just look for "Scenario 3").
To me, the main points in the more advanced modelling proposed in section 12.5 are:
1) for a given efficacy and difficulty pair the probability of giving the correct answer (ProbCorrect) is computed.
2) ProbCorrect is used to compute the binary PMF for the single question with sat.BinaryPmf(...). Assuming the raw score to be the same as the number of correct answers, the binary PMF has only values 0 and 1.
3) The pmfs for all the questions are summed together
Starting from above I simply introduced a different function to compute the "binary" pmf:
in PmfCorrect(...) to compute the distribution of correct answers given the efficacy
An additional modification was needed due to the fact that the raw score is not the number of correct answers anymore. On the one hand the outcome of Exam.Reverse(...) is the number of correct answers, while the outcome of PmfCorrect(...) is a raw score which now includes the penalties (e.g. for an exam with 53 correct answers and 1 wrong answer the raw score is 52.75). This mismatch is taken care of inside the Likelihood(...) method.
Did anyone try to include the penalties too? If yes, what did you do?
Hello!
After going through the two modellings of the SAT problem in chapter 12 (in version 1.0.9 of the book), I was wondering if it was possible to do a further step forward and include the penalties in the problem.
Here is what I did (full version is here, just look for "Scenario 3").
To me, the main points in the more advanced modelling proposed in section 12.5 are:
1) for a given
efficacyanddifficultypair the probability of giving the correct answer (ProbCorrect) is computed. 2)ProbCorrectis used to compute the binary PMF for the single question with sat.BinaryPmf(...). Assuming the raw score to be the same as the number of correct answers, the binary PMF has only values 0 and 1.3) The pmfs for all the questions are summed together
Starting from above I simply introduced a different function to compute the "binary" pmf:
I use this new function:
PmfCorrect(...)to compute the distribution of correct answers given theefficacyAn additional modification was needed due to the fact that the raw score is not the number of correct answers anymore. On the one hand the outcome of
Exam.Reverse(...)is the number of correct answers, while the outcome ofPmfCorrect(...)is a raw score which now includes the penalties (e.g. for an exam with 53 correct answers and 1 wrong answer the raw score is 52.75). This mismatch is taken care of inside the Likelihood(...) method.Did anyone try to include the penalties too? If yes, what did you do?