Stratification methods do not select the most informative item inside the stratum for a given examinee

[douglasrizzo]

When using one of the following item selectors: AStratSelector, AStratBBlockSelector, MaxInfoStratSelector, MaxInfoBBlockSelector; the method stratifies the item bank but does not select the item that is maximally informative for a given theta value inside that stratum.

This issue has already been fixed in both the master and dev branches, as well as in version 0.17.2 submitted recently to Pypi.

I'm just waiting for confirmation from the interested third-party that sent me the issue via e-mail to close it.

The snippet below was used to reproduce the issue, using an item bank of 5000 items. When the response vector is changed, the same item was selected both for a low-proficiency and high-proficiency examinee.

import numpy as np
from catsim.estimation import *
from catsim.initialization import *
from catsim.irt import *
from catsim.selection import *
from catsim.simulation import *
from catsim.stopping import *

items = np.genfromtxt("test.txt")

administered_items = [1, 2346, 17, 4444, 3490]
responses = [True] * len(administered_items)
# responses = [False] * len(administered_items)
print(f"Response vector is: {responses}")

initializer = FixedPointInitializer(0)
selector = MaxInfoStratSelector(test_size=2000)
estimator = NumericalSearchEstimator()
stopper = MinErrorStopper(.2)

new_theta = estimator.estimate(items=items,
                               response_vector=responses,
                               administered_items=administered_items,
                               est_theta=initializer.initialize())
print(f'Estimated Theta is: {new_theta}')

_, start_pointer, end_pointer = selector._get_stratum(items, 5)
organized_items = selector.postsort_items(items, est_theta=new_theta, using_simulator_props=False)
print(f"Items in current stratum are: {organized_items[start_pointer:end_pointer]}")

stratum_items = items[organized_items[start_pointer:end_pointer]]
print(f"Item information in current stratum for current theta is: {irt.inf_hpc(new_theta, stratum_items)}")

item_index = selector.select(items=items,
                             administered_items=administered_items,
                             est_theta=new_theta)

print(f"Selected item index is: {item_index}")