New Attempt Graph

[0815-xyz]

I feel bad about opening up an issue with the attempt graph right after it came out so brand new. But I don't know where better to voice my concerns or questions. Please guide me if I'm wrong. The old data table used to look like this: And it absolutely made sense to me to have the ability measure column being displayed after the column for the chosen question level and answer pattern since the measure is calculated based on them. Now the new data table looks like this: And firstly I want to say that I'm really happy it shows both the target and administered difficulty levels. But I'm also really confused about the meaning of the ability measure as the leftmost column. Does it represent the measure obtained after the previously answered question? Then where does the ability measure in the first row come from? In the attached screenshot it coincides with the minimum level of the activity settings (starting level 2, min 1, max 4) but when choosing different settings (e.g. starting level 3, min 2, max 4) it rather seems to mirror the starting level. And lastly why is it shown as an integer? Wouldn't a float (as before) be much clearer here? It also seems to have been rather truncated than rounded. I'm really glad about all the development that's going on with the plugin and would like some clarification here. Thanks in advance!

[vtos]

@0815-xyz

Hi, thanks for the feedback, it's always appreciated! Overall, I see what you mean.

Now, just an overview of how the new chart has been developed. We simply took the data for the old graph and used it to build the new one. We haven't changed how the data is obtained and formatted. So, the chart is built exactly the same way how the old one was built. As for the new table, it's just a representation of the chart data. Moodle builds such tables for all charts by default. My idea was it could actually be a replacement of the old table. The big difference between the tables which I see is that the ability measure value is the value you have BEFORE the question in the same row is answered. I believe the 'old' table contained the ability measure value AFTER the relevant question is answered. My personal opinion is the 'old' table's approach makes more sense, because there you can see the final ability measure value, where the test actually has stopped. In the new approach you don't even see this value. But this is how it was in the previous implementation of the graph - the table was built separately from it in the previous versions, so the graph and the table have different approaches to visualize the same data. I'll investigate it more to prove my thoughts above, because what I've written here is just off the top of my head for now. I'll get back with something sensible soon.

Thanks again for providing the feedback, it's really important for making the plugin better.

[0815-xyz]

Hi Vitaly,

thank you for the fast reply and the willingness to investigate. I absolutely agree with you that it makes more sense if the ability measure in the table is the one obtained after answering the question for that round.

As I actually thought that the graph was built on the table data (and not reversely the table data just being extracted by Moodle from the graph), I addressed the problem as being in the table itself. But the issue is actually not only in the table but also already reflected in the graph itself.

Old version:

New version:

One can see that the new graph shows wrong ability measures.

As it also makes total sense not to unnecessarily recalculate all values for table and graph separately, I think we might have to use the opposite approach and built the new graph not on the data of the old graph but on the data extraction that was used for the old table. If that is a viable approach or not I have to leave for you to judge as I unfortunately don't have enough insight into the code.

[vtos]

@0815-xyz Hi, I was wrong - this is how the new implementation strips out those values. Getting the raw values themselves is actually fine. After applying a fix I was able to get this result for something close to your configuration above:

It still has some differences in values compared to your example, but, perhaps, this is due to some slight difference in configuration for the quiz? I wasn't able to reproduce your config exactly, may be you have more questions in your bank for each level that matters for the algorithm, etc.

So, the fix will be included in the next version which is expected pretty soon.

[0815-xyz]

Hi @vtos ,

This absolutely fixes it and looks really great! I checked the differences between our graphs and realized that they stem from my settings having a difficulty range from 1 to 4 and yours having it from 2 to 4. This span (max level - min level) has an influence on the ability measure because it is used when transforming the data onto the logit scale for the calculations within the algorithm.

Thanks again for your fast solution! I'm as always looking forward to the next version.

[vtos]

Fixed on the master branch, will appear in the closest release.