campsych / concerto-platform

Concerto Platform - Open-Source Online Adaptive Testing Platform
https://concertoplatform.com/
Apache License 2.0
153 stars 88 forks source link

Help for polytomous CAT development #276

Closed mgonper closed 3 years ago

mgonper commented 4 years ago

Concerto Platform version

v5.0.9

Expected behavior

Like in the polyCAT version, I expected to set the category thershold by indicating the Andrich Thershold for each one of the item parameters (p1,p2,p3,p4)

Actual behavior

I can actually run the test without errors with "calculate theta" and "calculate SEM" unchecked but if I check it an error is displayed after answerig the first item "Test problem encountered. Execution halted"

Steps to reproduce the issue

Configure an item bank with 4 to 6 response categories and give them their categories according to our previous PCM analysis. CAT options: Minimum Accuracy 0.50, Item Parameters Number: 1, Model: PCM, Next item criterion: MFI

przemyslawlis commented 4 years ago

Required number of item parameters for PCM model is: response categories num - 1. You haven't pasted any logs, but judging from your settings (Item Parameters Number: 1), and number of response categories you're using (4 to 6) - this is the source of the problem.

Please paste your item bank CSV and error output if you're still having the problem.

mgonper commented 4 years ago

Thank you very much for your answer, I was trying with another bigger item bank but now I'm trying Concerto with the following item bank:

"id","question","responseOptions","trait","fixedIndex","p1","p2","p3","p4","p5","p6","p7","p8","p9" "1","En la última semana, ¿con qué frecuencia ha notado los ojos secos?","{""type"":""options"",""optionsRandomOrder"":""0"",""options"":[{""value"":0,""label"":""Nunca-0""},{""value"":1,""label"":1},{""value"":2,""label"":2},{""value"":3,""label"":3},{""value"":4,""label"":4},{""value"":5,""label"":5},{""value"":6,""label"":""Siempre-6""}],""painMannequinGender"":"""",""painMannequinGenderValue"":""male"",""defaultScore"":""0"",""scoreMap"":[{""value"":""0"",""score"":""0"",""trait"":""""},{""value"":""1"",""score"":""1"",""trait"":""""},{""value"":""2"",""score"":""2"",""trait"":""""},{""value"":""3"",""score"":""3"",""trait"":""""},{""value"":""4"",""score"":""4"",""trait"":""""},{""value"":""5"",""score"":""5"",""trait"":""""},{""value"":""6"",""score"":""6"",""trait"":""""}],""gracelyScaleShow"":""""}","","","-2.25","-1.5","-1","-0.25","1","1.5","","","" "2","Cuando sintió los ojos secos, en general, ¿cómo era de intensa la sensación de sequedad?","{""type"":""options"",""optionsRandomOrder"":""0"",""options"":[{""value"":0,""label"":""No lo he sentido-0""},{""value"":1,""label"":1},{""value"":2,""label"":2},{""value"":3,""label"":3},{""value"":4,""label"":4},{""value"":5,""label"":5},{""value"":6,""label"":""Extremadamente intensa-6""}],""painMannequinGender"":"""",""painMannequinGenderValue"":""male"",""defaultScore"":""0"",""scoreMap"":[{""value"":0,""score"":0,""trait"":""""},{""value"":1,""score"":1,""trait"":""""},{""value"":2,""score"":2,""trait"":""""},{""value"":3,""score"":3,""trait"":""""},{""value"":4,""score"":4,""trait"":""""},{""value"":5,""score"":5,""trait"":""""},{""value"":6,""score"":6,""trait"":""""}],""gracelyScaleShow"":""""}","","","-2","-1","-0.30","0.50","1.25","2","","","" "3","En la última semana, ¿con qué frecuencia ha tenido sensación de arenilla en los ojos?","{""type"":""options"",""optionsRandomOrder"":""0"",""options"":[{""value"":0,""label"":""Nunca-0""},{""value"":1,""label"":1},{""value"":2,""label"":2},{""value"":3,""label"":3},{""value"":4,""label"":4},{""value"":5,""label"":5},{""value"":6,""label"":""Siempre-6""}],""painMannequinGender"":"""",""painMannequinGenderValue"":""male"",""defaultScore"":""0"",""scoreMap"":[{""value"":0,""score"":0,""trait"":""""},{""value"":1,""score"":1,""trait"":""""},{""value"":2,""score"":2,""trait"":""""},{""value"":3,""score"":3,""trait"":""""},{""value"":4,""score"":4,""trait"":""""},{""value"":5,""score"":5,""trait"":""""},{""value"":6,""score"":6,""trait"":""""}],""gracelyScaleShow"":""""}","","","-1.75","-0.50","0.50","1.25","2","2.5","","","" "4","Cuando sintió como arenilla en los ojos, en general, ¿cómo era de intensa esa sensación?","{""type"":""options"",""optionsRandomOrder"":""0"",""options"":[{""value"":0,""label"":""No lo he sentido-0""},{""value"":1,""label"":1},{""value"":2,""label"":2},{""value"":3,""label"":3},{""value"":4,""label"":4},{""value"":5,""label"":5},{""value"":6,""label"":""Extremadamente intensa-6""}],""painMannequinGender"":""female"",""painMannequinGenderValue"":""male"",""defaultScore"":""0"",""scoreMap"":[{""value"":0,""score"":0,""trait"":""""},{""value"":1,""score"":1,""trait"":""""},{""value"":2,""score"":2,""trait"":""""},{""value"":3,""score"":3,""trait"":""""},{""value"":4,""score"":4,""trait"":""""},{""value"":5,""score"":5,""trait"":""""},{""value"":6,""score"":6,""trait"":""""}],""gracelyScaleShow"":""both""}","","","-1.9","-1.25","-0.6","0.2","1.3","1.8","","","" "5","En la última semana, ¿con qué frecuencia notó ardor o escozor en los ojos?","{""type"":""options"",""optionsRandomOrder"":""0"",""options"":[{""value"":0,""label"":""Nunca-0""},{""value"":1,""label"":1},{""value"":2,""label"":2},{""value"":3,""label"":3},{""value"":4,""label"":4},{""value"":5,""label"":5},{""value"":6,""label"":""Siempre-6""}],""painMannequinGender"":""female"",""painMannequinGenderValue"":""male"",""defaultScore"":""0"",""scoreMap"":[{""value"":0,""score"":0,""trait"":""""},{""value"":1,""score"":1,""trait"":""""},{""value"":2,""score"":2,""trait"":""""},{""value"":3,""score"":3,""trait"":""""},{""value"":4,""score"":4,""trait"":""""},{""value"":5,""score"":5,""trait"":""""},{""value"":6,""score"":6,""trait"":""""}],""gracelyScaleShow"":""both""}","","","-2.1","-0.9","-0.3","0.6","1.2","1.8","","","" "6","Cuando notó como ardor o escozor en los ojos, generalmente, ¿cómo era de intenso?","{""type"":""options"",""optionsRandomOrder"":""0"",""options"":[{""value"":0,""label"":""No lo he sentido-0""},{""value"":1,""label"":1},{""value"":2,""label"":2},{""value"":3,""label"":3},{""value"":4,""label"":4},{""value"":5,""label"":5},{""value"":6,""label"":""Extremadamente intensa-6""}],""painMannequinGender"":""female"",""painMannequinGenderValue"":""male"",""defaultScore"":""0"",""scoreMap"":[{""value"":0,""score"":0,""trait"":""""},{""value"":1,""score"":1,""trait"":""""},{""value"":2,""score"":2,""trait"":""""},{""value"":3,""score"":3,""trait"":""""},{""value"":4,""score"":4,""trait"":""""},{""value"":5,""score"":5,""trait"":""""},{""value"":6,""score"":6,""trait"":""""}],""gracelyScaleShow"":""both""}","","","-1.8","-1","0","1","2","2.7","","","" "7","En la última semana, ¿con qué frecuencia notó los ojos cansados?","{""type"":""options"",""optionsRandomOrder"":""0"",""options"":[{""value"":0,""label"":""Nunca-0""},{""value"":1,""label"":1},{""value"":2,""label"":2},{""value"":3,""label"":3},{""value"":4,""label"":4},{""value"":5,""label"":5},{""value"":6,""label"":""Siempre-6""}],""painMannequinGender"":""female"",""painMannequinGenderValue"":""male"",""defaultScore"":""0"",""scoreMap"":[{""value"":0,""score"":0,""trait"":""""},{""value"":1,""score"":1,""trait"":""""},{""value"":2,""score"":2,""trait"":""""},{""value"":3,""score"":3,""trait"":""""},{""value"":4,""score"":4,""trait"":""""},{""value"":5,""score"":5,""trait"":""""},{""value"":6,""score"":6,""trait"":""""}],""gracelyScaleShow"":""both""}","","","-1.4","-0.75","0","0.75","1.5","2.2","","","" "8","Cuando notó sus ojos cansados, generalmente ¿cómo era de intenso ese cansancio?","{""type"":""options"",""optionsRandomOrder"":""0"",""options"":[{""value"":0,""label"":""No lo he sentido-0""},{""value"":1,""label"":1},{""value"":2,""label"":2},{""value"":3,""label"":3},{""value"":4,""label"":4},{""value"":5,""label"":5},{""value"":6,""label"":""Extremadamente intensa-6""}],""painMannequinGender"":""female"",""painMannequinGenderValue"":""male"",""defaultScore"":""0"",""scoreMap"":[{""value"":0,""score"":0,""trait"":""""},{""value"":1,""score"":1,""trait"":""""},{""value"":2,""score"":2,""trait"":""""},{""value"":3,""score"":3,""trait"":""""},{""value"":4,""score"":4,""trait"":""""},{""value"":5,""score"":5,""trait"":""""},{""value"":6,""score"":6,""trait"":""""}],""gracelyScaleShow"":""both""}","","","-2.3","-1.50","-1","-0.50","0.25","1.3","","","" "9","En la última semana, ¿con qué frecuencia notó dolor de ojos?","{""type"":""options"",""optionsRandomOrder"":""0"",""options"":[{""value"":0,""label"":""Nunca-0""},{""value"":1,""label"":1},{""value"":2,""label"":2},{""value"":3,""label"":3},{""value"":4,""label"":4},{""value"":5,""label"":5},{""value"":6,""label"":""Siempre-6""}],""painMannequinGender"":""female"",""painMannequinGenderValue"":""male"",""defaultScore"":""0"",""scoreMap"":[{""value"":0,""score"":0,""trait"":""""},{""value"":1,""score"":1,""trait"":""""},{""value"":2,""score"":2,""trait"":""""},{""value"":3,""score"":3,""trait"":""""},{""value"":4,""score"":4,""trait"":""""},{""value"":5,""score"":5,""trait"":""""},{""value"":6,""score"":6,""trait"":""""}],""gracelyScaleShow"":""both""}","","","-1","-0.25","0.75","1.4","2.2","2.9","","","" "10","Cuando notó dolor de ojos, en general, ¿cómo era de intenso ese dolor?","{""type"":""options"",""optionsRandomOrder"":""0"",""options"":[{""value"":0,""label"":""No lo he sentido-0""},{""value"":1,""label"":1},{""value"":2,""label"":2},{""value"":3,""label"":3},{""value"":4,""label"":4},{""value"":5,""label"":5},{""value"":6,""label"":""Extremadamente intensa-6""}],""painMannequinGender"":""female"",""painMannequinGenderValue"":""male"",""defaultScore"":""0"",""scoreMap"":[{""value"":0,""score"":0,""trait"":""""},{""value"":1,""score"":1,""trait"":""""},{""value"":2,""score"":2,""trait"":""""},{""value"":3,""score"":3,""trait"":""""},{""value"":4,""score"":4,""trait"":""""},{""value"":5,""score"":5,""trait"":""""},{""value"":6,""score"":6,""trait"":""""}],""gracelyScaleShow"":""both""}","","","-1.6","-1.1","-0.4","0.3","1","1.8","","","" "11","En la última semana, ¿con qué frecuencia le picaron los ojos?","{""type"":""options"",""optionsRandomOrder"":""0"",""options"":[{""value"":0,""label"":""Nunca-0""},{""value"":1,""label"":1},{""value"":2,""label"":2},{""value"":3,""label"":3},{""value"":4,""label"":4},{""value"":5,""label"":5},{""value"":6,""label"":""Siempre-6""}],""painMannequinGender"":""female"",""painMannequinGenderValue"":""male"",""defaultScore"":""0"",""scoreMap"":[{""value"":0,""score"":0,""trait"":""""},{""value"":1,""score"":1,""trait"":""""},{""value"":2,""score"":2,""trait"":""""},{""value"":3,""score"":3,""trait"":""""},{""value"":4,""score"":4,""trait"":""""},{""value"":5,""score"":5,""trait"":""""},{""value"":6,""score"":6,""trait"":""""}],""gracelyScaleShow"":""both""}","","","-1.4","-0.8","-0.1","0.7","1.4","2.1","","","" "12","Cuando le picaron los ojos, generalmente, ¿cómo era de intenso ese picor? ","{""type"":""options"",""optionsRandomOrder"":""0"",""options"":[{""value"":0,""label"":""No lo he sentido-0""},{""value"":1,""label"":1},{""value"":2,""label"":2},{""value"":3,""label"":3},{""value"":4,""label"":4},{""value"":5,""label"":5},{""value"":6,""label"":""Extremadamente intensa-6""}],""painMannequinGender"":""female"",""painMannequinGenderValue"":""male"",""defaultScore"":""0"",""scoreMap"":[{""value"":0,""score"":0,""trait"":""""},{""value"":1,""score"":1,""trait"":""""},{""value"":2,""score"":2,""trait"":""""},{""value"":3,""score"":3,""trait"":""""},{""value"":4,""score"":4,""trait"":""""},{""value"":5,""score"":5,""trait"":""""},{""value"":6,""score"":6,""trait"":""""}],""gracelyScaleShow"":""both""}","","","-2","-1.3","-0.6","0.1","0.8","1.5","","",""

When I run the CAT, after answering 1 or 2 items, the following error is found:

"Test problem encountered. Execution halted."

And the error log is:

Test #38, R error. Error in dpr[i, x[i] + 1]: subscript out of bounds

Thank you very much for your help, I'll be waiting for your answer

Kind regards,

przemyslawlis commented 4 years ago

What's your current "Item Parameters Number" setting value? Have you changed it to match your selected model?

mgonper commented 4 years ago

Thank you very much for your reply,

I'm trying this platform with several item banks and now I'm noticed that the Item Parameters number is not and indicator of the Rasch model used (1PL, 2PL...) and it's used to indicate the number of threshold values. I followed the instructions for polytomous node and it recommended lo leave the first difficulty parameter. By doing so, and using the Andrich Thresholds, I left blak the parameter "1" and set the parameters "2" and "3" (for a three-category item); again, I found and error when choosing the bottom categories. I decided to change these values by the three "category measures" provided by Rasch analysis and now it seems to work fine.

Kind regards,

Mariano González Pérez Doctor (PhD) en Óptica, Optometría y Visión Profesor Asociado Facultad de Óptica y Optometría Universidad Complutense de Madrid marian06@ucm.es

La información contenida en este correo es CONFIDENCIAL, de uso exclusivo del destinatario/a arriba mencionado. Si ha recibido este mensaje por error, notifíquelo inmediatamente por esta misma vía y proceda a su eliminación, ya que ud. tiene totalmente prohibida cualquier utilización del mismo, en virtud de la legislación vigente. Los datos personales recogidos serán incorporados en la actividad de tratamiento 'Correoweb', bajo la titularidad del Vicerrectorado de Tecnologías de la Información, y en él el interesado/a podrá ejercer los derechos de acceso, rectificación, supresión u oposición ante el mismo (artículos 15-21 del Reglamento UE 2016/679 de 27 de abril de 2016). <> Antes de imprimir este correo piense si es necesario: el medio ambiente es cosa de todos. This message is private and confidential and it is intended exclusively for the addressee. If you receive this message by mistake, you should not disseminate, distribute or copy this e-mail. Please inform the sender and delete the message and attachments from your system, as it is completely forbidden for you to use this information, according to the current legislation. No confidentiality nor any privilege regarding the information is waived or lost by any mistransmission or malfunction. The personal data herein will be collected in the file "Correoweb", under the ownership of the Vice-Rectorate for Information Technologies, in which those interested may exercise their right to access, rectify, erasure or right to object the contents (article 15-21 of Regulation (EU) 2016/679, General Data Protection Regulation). <> Before printing this mail please consider whether it is really necessary: the environment is a concern for us all.

El vie., 24 abr. 2020 a las 15:03, przemyslawlis (notifications@github.com) escribió:

What's your current "Item Parameters Number" setting value? Have you changed it to match your selected model?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/campsych/concerto-platform/issues/276#issuecomment-618995365, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGM4T3XI2JCKKNXZS53AJADROGE3HANCNFSM4LG45OAQ .

mgonper commented 3 years ago

Coming back to this issue, I'm trying to use Andrich Thershold again. In I have 4 to 6 response options in my item bank, should I set Item Parameters to 5?

And, for example: I've colapsed two response options in one item: option 4 and option 5 has the same value , should I have to write, for example, thar p3=0.53 and also p4=0.53

Thank your very much for your help