Light up assigned exercises in-context

[rbeezer]

@oscarlevin wants to have exercise of a PreTeXt book, that are actually assigned, to be marked in some way that is clear to the reader at the location of the exercise in the text (in context). On a Runestone Server, the assignments for a course are created by the instructor and stored.

Analogous to the progress indicator, is there something that can go in every exercise that will allow it to "light up" for the reader if, and only if, it is in the database as assigned?

We know the "Runestone ID" of the interior (interactive) Runestone component. That could be placed in the enclosing PreTeXt exercise, PROJECT-LIKE, OR task somehow (HTML attribute? data-runestone-id?) and if that interior component is assigned, then maybe the problem number changes color, gets bigger, is in a box or circle, or something that screams "I am assigned!".

Obviously I'm glossing over the server-side details, but it seems very similar to me to the progress indicator, which is doing a per-student, per-page database query, it seems.

Might be a fiddle to push the Runestone ID outwards to the wrapper, so if probing for it is not hard, then...

[bnmnetp]

I think this is very do-able, with a bit more discussion.

1. We do some customization already when we know the student has entered the page from an assignment.
2. We should think more about which problems to highlight... All assignments? Might be confusing to highlight things that were part of an assignment that happened weeks ago? Or maybe informative if reviewing the chapter for test prpe to see the questions that were asked. Current Assignments? Where today is before the due date for a given assignment?

[oscarlevin]

I think there is a strong case for your first option of highlighting all assigned questions, even those that are part of past assignments. Like you say, this can help students focus on what their instructor thought was important when reviewing. If the assignment is old, then students can quickly see the reason they got a poor score when they see that a problem in that section was assigned put not completed. Plus, in my case I will often allow students to go back and complete old assignments for full or partial credit, and this will help them find what they missed when reviewing.

Will the problem show as "completed" somehow within the section? Thinking about how to display all this (and these are just ideas, not requests; any indication of a problem being part of an assignment would be fantastic):

1. In the left margin, problems that are assigned but not completed get a medium-large red or orange exclamation mark.
2. When an assigned problem is completed correctly, there is a green checkmark in its place.
3. For essay/reading questions, it is a light green (or grayed) checkmark until it is graded
4. Hovering over the mark gives a tool-tip: "In assignment [assignment-name]; due [due date]"
5. Clicking on the mark takes the student to the assignment page.

[bhoffman0]

I like these ideas! Showing a green checkmark or dot on all exercises when done, whether or not in an assignment, would be good too. Right now, students may see "9 out of 10 activities attempted" on a page, but they don't know which one they missed, and they have to go back and run them all to try to figure it out. It would also be great to have a score me button on each page like in assignments too to see if they did them all the exercises on a page correctly. There could be a stop light signal on each exercise; red not attempted, orange attempted, green attempted and done correctly.

[bnmnetp]

I'm surprised they need to redo them... If they have attempted almost any autogradable problem then the feedback is restored when the page is reloaded, which should make it pretty clear whether they had it right or wrong... Except for unit test data for coding questions.

Is this mainly an issue with activecode problems where we do not restore the unit test results? Maybe it is time to re-think what/how we restore those. We have not restored the code for activecodes except on-demand to keep page load times down, but perhaps with the new server we can revisit that.

These are good ideas!

[bjones1]

I agree with Beryl's idea -- I also have students complain that they aren't sure what problems they're missing, and seeing a checkmark for completed problems vs a dot (or any obvious graphical difference) would help communicate that.

On Mon, Jul 4, 2022 at 10:23 AM Beryl Hoffman @.***> wrote:

I like these ideas! Showing a green checkmark or dot on all exercises when done, whether or not in an assignment, would be good too. Right now, students may see "9 out of 10 activities attempted" on a page, but they don't know which one they missed, and they have to go back and run them all to try to figure it out. It would also be great to have a score me button on each page like in assignments too to see if they did them all the exercises on a page correctly. There could be a stop light signal on each exercise; red not attempted, orange attempted, green attempted and done correctly.

— Reply to this email directly, view it on GitHub https://secure-web.cisco.com/10-fJ92UlsjkpZBkoJzHg34n5XDDFFrHJ-FanIK3xK0Goj0lpbULqVyuo8r_VJO2KZADaFMHA_1xnLjt5OmVr17vFhRHV_-H7Tv-SyHNaTNyhF_Q8bM3PstpQmQnkA1PTeg-1Cns_2BSzbYJb2gnkxkizBfilKS7kD027gg5MZXI1SKdLmpB7OfThIBHDfT1dJAZcHuisOedehrZbJjbMIHgMaDE_OdoVBa3-OZFWqrgj03JAdoO7syXDqGSTXye0aryMGf1tcbjw95IgAi9R5_3q5D5rE2UXOMIGTuvrQE4iOrBbKIZrC4u3yKbIN0Fa/https%3A%2F%2Fgithub.com%2FRunestoneInteractive%2FRunestoneServer%2Fissues%2F1971%23issuecomment-1173935595, or unsubscribe https://secure-web.cisco.com/14JRGnlJWM6QIGihTcIndm0MAgtpDQSfWdFh_u3esqMEbqSO4iM_4VktuD9LIeddk5exi8Ve7UmzXkGqaRMXhFqZwa0YV-3ObB-Xvb8d8n0K651HLWAdGrjuUJ3yLwQMiByVGONYnpFiTLBC7MBtZXEOZAkmNln_A26kseLoe2RbR0zT4c6J64IoEZP_2RMGAR8XAnSoq5vSBcWrkUQRsVv4_RjrWIYcsJnl-eKdICqxuVLAnXZRnmNgUHeIcTy_Gqp11B7TeT6l51tQ6yPNfC2yjgJQlrY6NrbZd-zLtj1SvG7bXWKf4oYNge9dF7K8m/https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAQG6EDAAUHUS2FB4SHKACTVSL6X5ANCNFSM52RGWOHQ . You are receiving this because you are subscribed to this thread.Message ID: @.*** com>

-- Bryan A. Jones, Ph.D. Associate Professor Department of Electrical and Computer Engineering 231 Simrall / PO Box 9571 Mississippi State University Mississippi State, MS 39762 http://www.ece.msstate.edu/~bjones bjones AT ece DOT msstate DOT edu voice 662-325-3149 fax 662-325-2298

Our Master, Jesus Christ, is on his way. He'll show up right on time, his arrival guaranteed by the Blessed and Undisputed Ruler, High King, High God.

• 1 Tim. 6:14b-15 (The Message)

[oscarlevin]

I'm thinking of what happens when you try to submit a form and the site says you are missing a field or entered something not allowed: they usually clearly highlight which field is the problem. Yes, a user could look at all the fields and see that some are empty, but this is harder, so providing visual clues to what still needs to be done is very useful. I think students would like to clearly see when they are done with a question and when there is an assigned question they still need to do.

[bhoffman0]

It's a problem with activecodes, codelens, and video activities. They do not show if they were attempted on a page refresh. You don't know if you've attempted an activecode before unless you click on Load History for each one on a page refresh. If it's autogradable, the database knows at that point whether the student did it correctly too, so there should be a difference between attempted and correct if we're adding in visual cues.

[rbeezer]

Lots of good ideas, and better than the reply I was formulating. ;-) I think everybody is on to this one: color and shape is good for accessibility.

I was going to suggest past/present/future but that does not address completed/correct.

Oscar's "missed a field" analogy reminds me how the whole field gets outlined in red, and/or gets a red disc with a white X across it out in front.

[bnmnetp]

It is important for everyone to remember that there are multiple use cases to consider.

1. Some of the problems on a page may be part of a problem set that is being graded for correctness and some will not.
2. All of the problems on a page may be part of a reading assignment, where correctness is not an issue.
3. A teacher may have simply given verbal instructions to the students to read certain pages and do all of the activities on each page.
4. What if a question on a page is part of a reading assignment AND is assigned as problem to check for correctness?
5. Some things we know are correct / incorrect
6. Some things - videos/activecode/showEval do not have "correctness"

What I'm hearing is that:

We want to highlight problems that are part of problem sets so that as a student is reading they know that a particular thing has been assigned and they should try to get it right. - This is certainly the cleanest and simplest thing to implement.

We want to add an indicator to things like videos and other activities where there is not already obvious feedback that the reader has interacted with the thing at some point in the past. But.. for consistency do we want to have the same indicator for things where it is already evident?

For activecodes we could consider several options:

1. We could always restore the timeline and eliminate the "load history" button. - bigger server load but maybe worth a try...
2. If there is a unittest we could indicate their score on the unit tests near the activecode somehow
3. If there is history to load we could just give some indication that they had already tried to do something on this particular problem.
4. I do not think it would be wise to re-run every activecode on a page using their latest code so that output could be restored. I think that would be a killing blow to the JOBE servers.

This is definitely a time where doing some mockups would be useful.

This stuff always makes me feel like the grumpy old man 😤. Hey kids, I remember when we used to have to write down the problems we had to do in our notebooks, and then be sure we actually did the problem numbers we wrote down. How did we survive!? That said we have an opportunity to make the learning experience better for the students so we should do that if we can. Other commercial systems certainly do this kind of thing, so we should too.

[bnmnetp]

Here is one idea:

Others can help with colors sizing etc....

I was thinking that the red paperclip would indicate that this question is part of an assignment. A green checked box shows that the student has done it (and is correct correct if autogradable) An orange unchecked box would indicate it was not done yet or still incorrect

I don't think we want to have a whole bunch of different things for students to figure out and I don't think we want to clutter things with too many status icons...

[bhoffman0]

Love it!

[rbeezer]

Looking good! Yes, I'd been thinking 3 max. You can put a tooltip on each one so a reader can figure out what they mean once they start showing up? "Assigned", "Completed", "Pending"?

[rbeezer]

Thinking clearer. If assigned, always either green check or orange box? Making paper clip redundant?

What if orange box was "incomplete and due in less than a week" (or similar time period, configurable?). Then the paper clip would mean assigned, not complete, not on the horizon?

[bnmnetp]

In my mind I was thinking of this as two distinct questions:

1. Is this exercise part of an assignment?
2. Have I attempted to do anything with this exercise (especially for video/codelens/showeval) and if so is it correct. This way we can give indicators to students even if something is not assigned.

All of this is just me hacking html on a test page, so very little invested so far.

[github-actions[bot]]

Stale issue message

[github-actions[bot]]

Stale issue message

[github-actions[bot]]

Stale issue message

[Alex-Jordan]

Has this feature (the paperclip) been implemented?

[bnmnetp]

It has been done for webwork questions. Not for Runestone questions. But it mostly seems redundant for Runestone questions since we restore the feedback, so students know that they have answered the question. This would be relatively simple for a group of students in @pearcej class to take on.

[Alex-Jordan]

I don't see this happening in portlandcommunitycollege_orcca_fall23. For instance I have an assignment called "Section 1.1" which includes the exercise numbered 2 in that section of the book (which is a WW problem). But logged in as a student (my jordanstudent account) I do not see any indication in context that this exercise is part of an assignment. (I don't see the paperclip.)

[bnmnetp]

No, I have not implemented the paperclip, sorry. Only that the student has the question right or wrong.

[Alex-Jordan]

OK, I'm glad it's not the case that it's implemented and it's a bug that I don't see it :)

Do you have thoughts on how to implement this? I am wondering if the following is too crude: Upon page load, query through all "problem set"s, gathering a list of all ids of exercises that are in use in some "problem set". Including no doubt many exercise ids that are not even present on the current page.

Then construct a CSS snippet that styles based on those id's, and insert it into action. I'm wondering if that could be more efficient than visiting all of the exercises on the page and messing with their DOM structure.

Nothing urgent here. Just following up with my thoughts.

[bnmnetp]

The paper clip idea seems good in theory, but...

Should a paper clip show for all assignments? Assignments that are due in the next two weeks? Some instructors have all of their assignments for the semester queued up on day one. At some point this becomes more confusing because the student sees it is part of an assignment but has no idea of the due date.. If it is part of a review assignment, many teachers would not want students doing the problem ahead of time.

This could add significant load to the servers, and I'm kind of wondering if there is much to gain. The course page shows the assignments now, which should make it easy for students to click on an assignment and see the problems.

[Alex-Jordan]

I think with math courses, it would be a feature that helps a lot. Anecdotally I already had one student plodding along through all 100 or so exercises in context, instead of just doing the selected dozen that were assigned. In that case it's just that 1 in 20 or so students will inevitably have missed/forgotten/in-one-ear-out-the-other'd the guidance for how to do assignments.

But I understand what you are saying. Perhaps "problem sets" could have more structure? Right now, it looks like they have:

• due date
• visible to students boolean
• allow late submissions boolean
• timed assessment boolean
• time limit for timed assessment
• peer instruction boolean
• show async boolean
• description

I could probably suggest a dozen more useful attributes, based on helpful things I've seen in other platforms. For this discussion, additional dates and a boolean could help address what you are bringing up:

• open date (before this date, students cannot access the assignment, defaults to beginning of time)
• close date (comes on or after due date, and removes access to the assignment, defaults to end of time)
• show in context boolean (between open date and close date, show that paperclip in context for the exercises in this set)

[bnmnetp]

Thanks Alex,

Lets get all the configuration stuff on the table...

To be up front with my bias, I have an allergy to too much configuration, as I think it is the downfall of many systems, and adds complexity and maintenance cost going forward. But I'm willing to consider all the options.

[Alex-Jordan]

Are you asking me to suggest more config options I have seen in other platforms? The ones I think would be relevant for the "paperclip" are the three I mentioned. "Open date" and "close date" also help an instructor prevent the student from being overwhlemed with a huge list of assignments at any one given time.

Other config options I can think of don't seem relevant to this thread, but I could share somewhere. A new issue? I just don't want to confuse me sharing all of them with me lobbying for them in RA.