Hill Climbing Estimator contains dead code.

[matkg]

if set(response_vector) == 1:
            return float('inf')
elif set(response_vector) == 0:
            return float('-inf')

These statements will always return false.

[douglasrizzo]

Thanks for the report. I have proposed a fix, which I'll test further later, that does what was expected of the original code.

[matkg]

Yes that should work from my experience. But why returning -inf or plus inf and not just bmax or bmin?

[douglasrizzo]

By definition, if a student answers all items right or wrong, the test is incapable of measuring their true abilities, because they go beyond the reach of the test. The correct thing to do would be to have more items in the test, with higher and lower b values.

Returning bmax or bmin wouldn't be accurate, as users would think catsim has estimated that value when in fact we're just returning something because the test doesn't contemplate a student's proficiency, which is a flaw in the test.

My solution was to look at the theory behind maximum likelihood estimators. In these edge cases, they return +infinity or -infinity, so that's what I implemented.

[matkg]

Good explanation thank you. I know the theory behind it and it totally makes sense. Only that -inf or +inf would break my implementation in c# because of overflows. Thats why i use your dodd method for getting a corrected theta if all are true or false. Then I dont have to bother returning -inf or + inf right?

[douglasrizzo]

I believe Python represents infinity as a separate object, not as a number, so it avoids overflow problems. For example float('inf') + 1 == float('inf') is True.

As for the Dodd method, I have seem others works in the literature that employ or have employed it at some point, for example, in the ASVAB. I don't know if Dodd's has any mathematical meaning, but it seems like a reasonable heuristic.

Another solution would be to use a Bayesian estimator, which avoids the +inf/-inf problem, but I have never implemented one of those.