Knewton / edm2016

Code for replicating results in our EDM2016 paper
Apache License 2.0
59 stars 32 forks source link

Running without Tox #2

Open josephtey opened 7 years ago

josephtey commented 7 years ago

Hi there, I'm very interested in implementing IRT; this repo seems to be well-suited to what I am to achieve. However, i find it very difficult to play around with the code when I have to continuously run the command tox after any changes. What would be the best approach to achieve the same results WITHOUT using tox? What files should I individually run?

I also hope to implement this in a web application in the future, so knowing which files to run (without tox) would help as well.

Thanks.

khwilson commented 7 years ago

Here tox is used to do three things: setup virtualenvs, install requirements, and run tests. You can mimic this by creating a virtualenv yourself (virtualenv venv && source venv/bin/activate && pip install -r requirements.txt && pip install -e .) or you can use tox --develop to do a pip install -e . of this package into tox's virtualenv.

josephtey commented 7 years ago

Hi Kevin, Thanks for the prompt response; I've figured my way around Tox but I am still struggling to get the IRT parameters. I've been trying to get ability, and difficulty parameters for each student and each item with the assistments test dataset (test_run_irt.py), and after a bit of digging, in online_cross_validation.py, the parameters seemed to be referenced through .param_data, as well as the prob_recall. I've printed the prob_recall and .param_data, however, they don't seem to corellate in anyway to the data provided.

Here is what I'm working with:

printed the loaded data order_id user_idx item_idx correct time_idx
33022537 0 0 1 33022537 33022709 0 0 1 33022709 35450204 1 0 0 35450204 35450295 1 0 1 35450295 35450311 1 0 0 35450311 35450555 1 0 1 35450555 35450573 1 0 1 35450573 35480603 1 0 1 35480603 34288611 2 1 0 34288611 34288611 2 1 0 34288611

printed prob_recall (from online_cross_validation.py) [ 0.70205542 0.43549409 0.59171247 0.4637502 0.55909128 0.62561765]

printed iter_test_node.param_data (from online_cross_validation.py) {'thetas': array([[-0.21005269]]), 'offset_coeffs': array([[ 0.53032134], [-0.53032134]])}

I can't seem to derive a relationship between the parameters and probabilities with the above dataset. For instance, what is the ability for user 1, and the difficulty of item 0?

Thanks so much, looking forward for your response.