In LTI Advantage, pass actual scores back (with total possible)

[Alex-Jordan]

So...I've done zero testing with this. Because I'm not set up to test on Canvas or Moodle, and I can't really test with D2L without just diving in and trying things in production.

But my intent here is to stop sending the LMS decimal/percentage scores. For example if a student has 1/12 on an assignment, WeBWorK sends back 0.08 as the score (rounded down from 0.8333....) If the LMS grade item is set to also be out of 12 points, then the student sees they have 0.96/12, since 0.96 is 8% of 12.

So the intent here is that we send the LMS the 1 and the 12. Although I have not tested, this seemed surprisingly easy to do. The scoring utilities are already set up to return arrays with things like the 1 and the 12. @drgrice1 can I impose on you to critique this and if it's workable, try it out on Canvas and Moodle?

Related to this, I'd like to change content item selection to not use a scoreMaximum of 100 by default. Or at least make that configurable to use either 100 or the item's actual total. But that would be a separate PR and is perhaps controversial if you prefer it to be out of 100. But we are finding in practice that step #1 when connecting a WW course to a D2L course is to go change all the 100's to the actual assignment totals. I mention it now since it is somewhat related.

[drgrice1]

I have tested this pull request with Moodle (after fixing the $scoreiGiven typo), and I think this is great. I haven't yet tested with Canvas, but I suspect this will be correct for that also, and will fix an issue with rounding of scores that I have had for a long time.

I changed my mind on the content item selection max score. I think that the right thing to do is in fact to use the sum of the problem values for the max score. The instructor can always change that if needed. Blindly using 100 for the max score doesn't really make sense.

[drgrice1]

I also think that this should be implemented for LTI 1.1. I am rather certain that it supports this as well. I will investigate when I have more time.

[drgrice1]

I have now also tested this with Canvas. I have also implemented setting the max score for content item selection to be the sum of the values for the problems in a set.

I noticed that assignments created via the content item selection endpoint are not set to open in a new window by default for Canvas. That can also be set by parameters sent via a content item request. That should probably be added to. Perhaps as an option that is selected when the assignments are selected via the content item selection iframe page.

[drgrice1]

Apparently you can't actually specify if an assignment opens in a new tab or is embedded in an iframe via the deep linking specification. Unless I am missing something (see section 3.2 and the deep linking response examples in appendix B of https://www.imsglobal.org/spec/lti-dl/v2p0). You can specify how it opens in the various ways it can be open, but not which way to use.

Moodle makes this easy by allowing you to choose a default for the tool in general. Canvas doesn't have such an option it seems.

[drgrice1]

It seems that LTI 1.1 does not support the maximum score for grade pass back. However, we could slightly improve on rounding error by not rounding to two decimal places. I think that four decimal places would be better.

[Alex-Jordan]

In D2L, the content item selection links have always been "born" with the "Open as External Resource" flag checked. I'm unsure if this is a site default that the D2L admins have set, or if it is just D2L default in general.

It seems that LTI 1.1 does not support the maximum score for grade pass back.

That was my recollection from some thread I monitored long ago when people like @taniwallach and Mike and others were trying to address the rounding issue. I wonder if Tani recalls any reason in favor of only rounding to two decimal places.

[Alex-Jordan]

I fixed that typo and pushed. But did you want to just open a new PR with all the related things you mention having done?

[drgrice1]

Yeah, I can open a separate pull request. They are related, but work independent of each other.

[Alex-Jordan]

Note that you should restart the webwork2-job-queue for this. After previously having this work, I started seeing decimals again and it took me a while to figure out that was the issue.

[drgrice1]

Generally, any time the webwork2 code changes you should restart the job queue.

[Alex-Jordan]

I am suddenly concerned that this will cause problems if there is a set, and for one user the total weight of of the set is different than it is in the global set. The weight of an exercise can be overridden for one user. This is not uncommon, since two of the achievement rewards do this.

I don't know enough about how the LMS works to understand what would happen here. It seems like the grade item in the LMS has a point total that cannot differ from student to student. So would the student with a modified total weight just have their score adjusted proportionately?

[drgrice1]

From what I have seen with Canvas and Moodle, when the LMS receives this information, it adjusts the score to fit into the setting that it has. In other words it adjusts the score proportionately. I don't know what D2L does though.

I tested in Moodle and set the maximum score in Moodle for an assignment to 80, while in webwork the maximum score for the set is 100. I then submitted the answer for a problem which put my score for the set at 3.5 out of 100 on that set. So with this pull request 3.5 was the scoreGiven and 100 the scoreMaximum. I then looked at the grades in Moodle and it showed the score as 2.8 out of 80. So it did adjust the score proportionately.

I tested the same thing in Canvas, and saw the same result.