Assignment status is confusing to students

[bnmnetp]

A bit of background and history.

In summer of 2020 we added a feature to help students keep track of assignments better. The list below can get quite long in a class with lots of homework.

The status shown is advisory only and meant for students. It also pacified a lot of instructors who could not wrap their heads around the fact that we didn't have an official Submit button for a student to "submit" their work.

The solution we cam up with is below:

Students only need to click the button to mark this assignment in progress and once in progress a second click marks it as done. This really didn't (and still doesn't) seem like too much to ask.

The Problem

Unfortunately the button does not seem well placed, and students and instructors expect that when a student answers a question that we should automatically mark the assignment in progress.

Possible Solutions

1. Modify the assignment selection page and replace the cell, labelled A, for each assignment with a dropdown menu. This dropdown would have the options Not Started, In Progress, and Complete. Once the assignment was graded and grades were released this dropdown would be replaced with the % score as seen.
2. Move the button labelled B to the top of the assignment page or somewhere else more prominent to make it obvious to the student to click it when they are starting the assignment.

3. When we build the assignment selection page we could look at assignments that are not started and detect if any of the questions in said assignment have a an answer. At that point we would mark it as "In Progress"

   • This does not remove the need for a button so that a student can mark the assignment as finished.
   • This could also produce some surprises in students as they may not have even looked at the assignment but it could still be marked as in progress if they had answered a question while reading the text.

I would welcome thoughts from @oscarlevin @ericsonga @bhoffman0 @Alex-Jordan

[Alex-Jordan]

When an exercise is loaded (either in the book or in an assignment) if the student submitted an answer previously (either in the book or in an assignment) does their previous answer show?

Does the answer to that question depend on thing like it being a timed assignment?

On Sat, Dec 10, 2022, 2:03 PM Bradley Miller @.***> wrote:

A bit of background and history.

In summer of 2020 we added a feature to help students keep track of assignments better. The list below can get quite long in a class with lots of homework.

[image: Screenshot 2022-12-10 at 3 46 31 PM] https://user-images.githubusercontent.com/51115/206876552-0a5d97cc-b4f6-4715-95fc-11c4411a9219.png

The status shown is advisory only and meant for students. It also pacified a lot of instructors who could not wrap their heads around the fact that we didn't have an official Submit button for a student to "submit" their work.

The solution we cam up with is below:

[image: Screenshot 2022-12-10 at 3 46 58 PM] https://user-images.githubusercontent.com/51115/206876574-0f7f8fe4-017d-43fc-9a55-6efc89a102ab.png

Students only need to click the button to mark this assignment in progress and once in progress a second click marks it as done. This really didn't (and still doesn't) seem like too much to ask. The Problem

Unfortunately the button does not seem well placed, and students and instructors expect that when a student answers a question that we should automatically mark the assignment in progress. Possible Solutions

1.

Modify the assignment selection page and replace the cell, labelled A, for each assignment with a dropdown menu. This dropdown would have the options Not Started, In Progress, and Complete. Once the assignment was graded and grades were released this dropdown would be replaced with the % score as seen. 2.

Move the button labelled B to the top of the assignment page or somewhere else more prominent to make it obvious to the student to click it when they are starting the assignment. 3.

When we build the assignment selection page we could look at assignments that are not started and detect if any of the questions in said assignment have a an answer. At that point we would mark it as "In Progress"

• This does not remove the need for a button so that a student can mark the assignment as finished.
  • This could also produce some surprises in students as they may not have even looked at the assignment but it could still be marked as in progress if they had answered a question while reading the text.

I would welcome thoughts from @oscarlevin https://github.com/oscarlevin @ericsonga https://github.com/ericsonga @bhoffman0 https://github.com/bhoffman0 @Alex-Jordan https://github.com/Alex-Jordan

— Reply to this email directly, view it on GitHub https://github.com/RunestoneInteractive/RunestoneServer/issues/2032, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABEDOAGAZA7I67BIVTTEGX3WMT43DANCNFSM6AAAAAAS2QWE2Y . You are receiving this because you were mentioned.Message ID: @.***>

[bhoffman0]

@bnmnetp: I like the drop down idea in 1 and to automatically mark them as in progress in 3. I think the surprise part is fine. I like moving the button to the top in 2 too, but why not just make it a drop down like in 1? Then, students know any option is fine. I have to keep explaining that it toggles through 3 options and they can go back.

I think this still doesn't remove some of the confusion with this feature though. The problem that kept coming up for me was that 1 student would ask me to grade theirs before the due date because they were finished and were curious about their grade, and that would give everyone a "This assignment has been submitted" message and remove their options to toggle through the 3 states, and some would get stuck at the Not Started state. Then, I would get all these confused students telling me they didn't know how it got submitted when they were still working on it.

@Alex-Jordan yes, previous answers are shown for any question id that they've seen before. You could always make a copy of the question with a new id if you want to put it in an assignment or quiz without showing their previous ansers.

[Alex-Jordan]

I think the term "status" contributes to confusion. Can we brainstorm alternatives? Something that says "this is your progress so far" with less risk of saying "this is your grade on the assignment".

Here's a proposal. And forgive me if I sill am not seeing some things about this, the history, the technical challenges...

• When a student reaches the assignment selection page, use the "in progress" flag as now to report their "status". No change to current behavior (unless you like my suggestion to change the term "status")
• When a student actually loads the assignment and it is currently "not started"....
  • If the student has done nothing for that assignment yet (neither in the assignment or while in the book), then while still on that assignment page, if they submit an answer, that flags the assignment as in progress.
  • If the student had actually answered at least one question while reading the book, then we are already loading their previous answers now while showing them the assignment's page. If some answer is fetched at that time, prompt the student: "You have answered some of these exercises already in another place; flagging this assignment now as in progress."

[Alex-Jordan]

In other words, let them see that an assignment is "not started" while viewing the assignment selection page. But in some cases, as soon as they actually visit that assignment, it is auto-flagged as "in progress".

[bnmnetp]

Alex, to answer your first question, yes answers are restored when the student is looking at the question in context or on the assignment page. But NOT if the question is part of a timed assessment or if the question is seen as part of the practice feature.

[bnmnetp]

I'm certainly not wedded to the word status. As a programmer it makes sense, but maybe not as a student. Other labels that come to mind: progress? disposition? plight? condition?

I have resisted automating the "in progress marker" because it is a reasonably large amount of work, and I guess I'm not convinced there is very much benefit. Shouldn't we be teaching students to take some responsibility for getting their work done and organizing their time? Your proposal above might be a decent compromise, But my experience is that anytime we try to make question more stateful we increase the complexity in ways that is hard to imagine until things go wrong. I also think it is not that much effort for a student to click the button if we can get the interface right to make it obvious to them what they should do.

A similar approach would be to have the server check when a student clicks into the assignment. If the assignment is in the "not started" state but we detect that a student has answered one of the questions for that assignment we could set a flag to prompt that student about changing the state to "In Progress"

The student is still going to have to decide when to click the button to indicate that they think they are done. It is very hard to know automatically when they are done, especially if some questions require manual grading.

[oscarlevin]

I like all three suggestions and think each helps in its own way. Perhaps it would be worth thinking about a more general approach to the two "statuses" that we are discussing.

One is communication between runestone and the user (generally student) letting them know that their work has been saved, whether they have open assignments, etc. This definitely occurs on the checkmarks in TOC that get filled in automatically when enough of the elements on the page have been completed.

The other is communication between instructors and students (or students and themselves) to keep track of what they have done and what they still need to do. This is the model used for assignments we are discussing here.

I think it would be helpful to clarify that these are two independent features of runestone. It is possible for students to mark (for themselves) that they have tried as much as they are going to on a particular problem, and to tell their instructor it is ready to be graded. It is possible for students to set a reminder for themselves to ask a question about this in class. But a separate feature of runestone is that it tells the student how many times a question has been attempted and whether any of those attempts are correct.

Maybe calling these fields "Student Notes" and have messages like "I'm still working on this" or "I'm done, ready to be graded". Something that would be really helpful would be a note like, "Please regrade" to let the instructor know that they have revised the assignment (perhaps late). These could be on a dropdown as @bhoffman0 suggests.

It would be great to have this on a per-question level too. So a student can mark that they are satisfied with their work on a problem, or a reminder to come back to it.

[oscarlevin]

BTW, per-question status is also needed when questions in context should be marked as part of an assignment.

[bnmnetp]

@bhoffman0 Students can use the self-grade button. Its kind of crazy that they want you to do that for them. I will have to check the code, I didn't think that the status was locked until the grades were released. What was the reason that students didn't just press the self-grade button? Also, for anything that is autogradable the student gets immediate feedback. I feel like I'm missing part of the puzzle on this one.

[Alex-Jordan]

The homework systems that I have experience with (WeBWorK, Edfinity, MyOpenMath, WebAssign, a few more) don't distinguish progress on an exercise set from the overall score. Even zyBooks, which has a lot in common with Runestone. As a student, you don't have to understand the difference between making progress and submitting for a grade. That approach makes some of the issues being discussed here dissolve away.

But Runestone separates these ideas (progress and submitting for a grade). Is that an intentional decision? I can see it naturally coming about if you model homework process after pencil-and-paper homework: the student is "in progress" while they write stuff at home, and then there is this special moment where they hand it in. But imo it takes advantage of the online format to merge these concepts. Unlike working on paper at home and the handing something in at class time, while the student is doing things on a web site, they don't (by default) mentally separate the site (where they can be "in progress" on a set) from the instructor (to whom they turn it in).

[bnmnetp]

@Alex-Jordan -- But Runestone really does not. This progress stuff is just there because a bunch of teachers complained that their students could not keep track of what they had to do, AND the teachers didn't understand the model. They could not get around the fact that there was not submit button for students to say "I'm done" !

We do not need a student to submit their assignment we just keep recording all of their work and the teacher decides to grade whatever they want. Work done before the deadline, work done after the deadline we still keep track of everything, and everything is timestamped so if there is ever any question or confusion you can see exactly what happened when.

We clearly need a better way to explain that this functionality is ONLY there to help students organize themselves.

[bnmnetp]

@Alex-Jordan are you suggesting that we should just show the current score? If they have a 0 that is a pretty good indication that they haven't started? Or show the number of questions attempted?

I'm not sure if these other homework systems allow for problems that need manual grading but that has always been the difficulty... students see they are getting only 80% and freak out because the instructor hasn't had time to manually grade a short answer question or a turtle graphics program that does not lend itself to unit testing.

[Alex-Jordan]

Maybe we're talking in circles here because of ambiguity of some of the critical words in play. The verb grade in particular has been used in this thread in ways that to me, imply contradictions. Even with the same person using it. But I suspect it's because sometimes it's used to mean a specific action an instructor takes (or doesn't take) while managing a RS course, and other times it's being used more abstractly like factoring in a number into your offline grade book.

are you suggesting that we should just show the current score?

Yes, I think so. But qualified to calm down angsty students. In a way that communicates things like "But don't worry, the due date hasn't passed yet and you still have time to work on this". Or "Even though it's past the due date, you could still submit some answers and your instructor may or may not honor those submissions."

I'm not sure if these other homework systems allow for problems that need manual grading but that has always been the difficulty... students see they are getting only 80% and freak out because the instructor hasn't had time to manually grade a short answer question or a turtle graphics program that does not lend itself to unit testing.

At least some of those systems do allow manual graded exercises. And yes, it's a concern when the score is 0 but only because an instructor hasn't gotten to manual grading yet. WeBWorK behaves this way. However it's on my wish list for WeBWorK 3 to separate out what needs manual assessment so you qualify the messaging to the student. Something that communicates:

• "Currently your score on this assignment is 90% considering all automatically graded exercises and all of the manually graded exercises that have already been scored. However that does not account for a portion of this assignment that haave yet to be manually graded."
• Or "All exercises in this set must be graded manually, and none have yet been scored."
• Or "Currently your score on this assignment is 90%." in cases where there are no manually graded parts.
• Or "Your score on this assignment is 90%." in cases where there are no manually graded parts and the due date has passed. (So as to not suggest you are entitled to it possibly being raised.)

I know all these things I'm tossing out in quotes are highly verbose. At least for standalone WeBWorK, especially version 3, I am interested in streamlining how to deliver those kinds of messages without making them become distractions because they are so many words on the page.

[bnmnetp]

@Alex-Jordan you have put your finger on a long standing problem. In my recent instructor survey the number one "complaint" about Runestone was that the assignment/grading interface was confusing. Probably that is partly the interface and partly the language. It would be great if we could collectively clear that up.

1. Instructor: Creates an assignment
2. Student: works an exercise (also called problem or question ...) then clicks a button, that button may be labelled "Check Me", "Check", "Save" or "Save & Run"
3. Runestone: (grades, scores, checks, evaluates)? the exercise and gives the student feedback (Correct, Incorrect and if incorrect here is a hint). These hints are not to be confused with the hints that you can create separately in PreTeXt. Runestone also saves the latest answer along with an indication of whether the latest answer was correct or not to its database.
4. Student: May press the "Score Me" button at the bottom of the assignment page. (Note the caveats are already there)
5. Runestone: Grabs the (first, last, best) answer from the database for each exercise and (grades or scores) the exercise. This step goes beyond the correctness of the problem and actually assigns some number of points to the student for this exercise. (footnote 1)
6. Instructor Goes to the grading tab of the interface and selects this assignment On this page they can click on the Autograde and Display Totals button,
7. Runestone: performs the exact same operation as it did in 5 for every student (or some subset selected by the instructor) except that it records the (scores, grades) with a flag that lets the system know that this was done by the instructor and not the student. After the instructor has done this the grades are available in the gradebook
8. Instructor: presses the "Release Grades" button so that the student will now see their grade for this assignment. This step was added to reduce student angst when an assignment is only partially graded.
9. Instructor: May also select a particular exercise (Labelled Question) on the grading page and manually grade that exercise for one or more students.

Given all of that I'd like to hear from you all how we can clean up our use of language and our interface to make it less confusing.

NOTE In none of this did I describe any use of the button that changes the status of an assignment. Because it does not impact any of the functionality I described.

footnote 1: I know you may ask why not assign points in step 3? Step 3 happens locally, may happen in context, an exercise may be included in more than one assignment with different points values in each. Runestone didn't even have the concept of an assignment until several years into its life.

[pearcej]

I wonder if part of the confusion that the student's status is being reported as a their in-progress grade, rather than percent attempted? I wonder if it might be clearer for students if there were two columns, one with percent-attempted and the other with in-progress-grade (which would only appear when a button was pushed.) I suspect this might also create different confusion, but I thought I would put it out here.

[bnmnetp]

A couple of concrete things I would like to try in the short term

1. Change the choose assignment page so that the status column is labelled "progress"
2. Change the cells of the progress column to contain a dropdown menu with. the options "Not Started", "In Progress" and "Complete"

3. Change the wording of the progress messages on the assignment page

   1. Change "This assignment has not been started" message to "Click to mark this assignment as In Progress" where In Progress is a button.
   2. Change "This assignment is in progress" to "This assignment is in progress, click to mark it "Complete" where Complete is a button
   3. Change "This assignment has been submitted" (eek!!) to "This assignment is Complete click to mark it as "In Progress"

I think eliminating that word submit would be a good thing, and this interface makes it really clear that it is up to the student to take an action to change the status.

[oscarlevin]

What about even more explicit on the language that it is the students marking of the progress?

For 3.ii: "You have marked this assignment as In Progress, click to mark it 'complete'"

3.iii: "You have marked this assignment as Complete. Click to mark it as 'In Progress'"

Active voice and true.

Another easy (I think) fix would be flash a "attempt saved" message each time a student hits the "check", "run", etc button on a question.

[bjones1]

I like these changes. They make sense to me.

Here's a longish meditation on the current system, partially just to think through what we're doing. Does this make sense to you? What's the best way to explain this?

1. Most exercises (aka problems/questions) provide a student with immediate feedback, by marking a problem as correct or incorrect and providing addition feedback (a hint if incorrect, a reinforcement or observation if a problem is correct). This is a familiar concept for most students, since almost any online exercises system does the same. Providing this immediate feedback is IMHO one of the core strengths of Runestone.
2. Most exercises are intermingled with the surrounding instructional text. IMHO, this is another core strength of Runestone and is one of its unique features in the digital domain (I'm only aware of zyBooks with a similar approach). This is familiar to students, since traditional textbooks typically include example problems intermingled with the text.
3. The Runestone model defines assignments as an instructor-selected set of exercises with a due date. A given exercise may be included in zero, one, or many assignments. This is mostly familiar to students -- not all questions in a traditional textbook are graded.
4. The point value of a given exercise is set in the assignment, meaning that a given exercise may have no point value assigned, a single point value assigned by the one assignment it's included in, or multiple point values from multiple assignments. Because assignments have due dates, these points value vary with time -- exercises completed after the due date may receive no points. Therefore, a Runestone exercise cannot be "graded" -- have a point value assigned -- outside the context of an assignment. This is complex and where students probably get somewhat lost.
5. Runestone eliminates the concept of assignment submission. This is foreign to most students and instructors. We need some way to help students/instructors understand this.

Students want to know:

1. What questions are in the current assignment? Runestone's answer: the assignments page for a given assignment shows all exercises in that assignment. However, it currently lacks a way to indicate which exercises are in a given assignment when a student is reading the text of the book.
2. Have I completed all the questions in the current assignment? Runestone's answer is again the assignments page, but requires a student visually scan all problems on that page looking for any unanswered problems. Students have complained that it's easy to overlook an unanswered question in a long assignment.
3. Are the answers to any of these (auto-graded) questions wrong, meaning I need to correct these answers before the assignment is due? Runestone's answer is the same (scan the assignments page), with the same problems (overlooking an incorrect answer).
4. How do I submit my assignment? Runestone's answer (quoting The Matrix): there is no spoon.
5. Can/how do I turn in an assignment late? Runestone's answer: exercises may be worked at any time; instructors may choose to grade these. (I'm not aware of a way to auto-grade for reduced credit, though.)
6. What's my grade on this assignment? Runestone's answer: before an assignment is graded, the "Score Me" button; after the assignment is graded, the grade is shown.

Bryan

On Mon, Dec 12, 2022 at 8:56 AM Bradley Miller @.***> wrote:

A couple of concrete things I would like to try in the short term

1.

Change the choose assignment page so that the status column is labelled "progress" 2.

Change the cells of the progress column to contain a dropdown menu with. the options "Not Started", "In Progress" and "Complete" 3.

Change the wording of the progress messages on the assignment page

1. Change "This assignment has not been started" message to "Click to mark this assignment as In Progress" where In Progress is a button.
   2. Change "This assignment is in progress" to "This assignment is in progress, click to mark it "Complete" where Complete is a button
   3. Change "This assignment has been submitted" (eek!!) to "This assignment is Complete click to mark it as "In Progress"

I think eliminating that word submit would be a good thing, and this interface makes it really clear that it is up to the student to take an action to change the status.

— Reply to this email directly, view it on GitHub https://secure-web.cisco.com/1HXsWJ9cEZGB6MtIQscD2LGuOrbyAgSs1v9IYXfiA3yRpfDHNg1jbchGtODcnCR5yuW8Bsk9kP9ZLVI8cBzgBFbONnT-fJXLOIEp1APiT3De9F4y1VX9Ym2-dwcwRacgckWAAEJYhEShk7H2cwm2NoltkJm7FRwBu8vOJLWT2k4W-HXQT9zKR4UznntUbmi_3GV878dV9OUi2hsa3OtgQhrN2zk14bfkY0opfpnVsdVfmzLFqRaOGasG6EI9wgcSSopea-QCF0SmdeFKMUJuq_H0sfPWjux1tZEwPsTR4wPbZ0XdC131O4xAs7CrVbBX-/https%3A%2F%2Fgithub.com%2FRunestoneInteractive%2FRunestoneServer%2Fissues%2F2032%23issuecomment-1346647423, or unsubscribe https://secure-web.cisco.com/1m2UhNw7oSwvrsP-qOKacI5tgEr6p2dlgWdpbR4iR0UlkXMrfPKYet0oMFVI3Ce_7aI5Vy8RcgGQFdx1SUgNZ5VsGrjI6537UQTpEXgraK_Lc-Dwm1Eix1lAjj5VUTQ8jboRmCa-mngfdS-aOsAzoX4cPDaIOrPagVTsNFUHxROMHur7g8Q5B98nonHoPkcPp6CtjDu1RFSN2z5vmr9gpvowTbJboWQdtPYZUAON0jsL54Bi5lXGA_tatxp6wQRTaeuG_-wOGG12Hm8Kl3uyCVDUVCCLfHMWM5xIjzvQ6vUBX4uAszvnayo8T89s2-WQF/https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAQG6ECFJJXRDRD4U6GOEPLWM44IZANCNFSM6AAAAAAS2QWE2Y . You are receiving this because you are subscribed to this thread.Message ID: @.*** com>

-- Bryan A. Jones, Ph.D. Associate Professor Department of Electrical and Computer Engineering 231 Simrall / PO Box 9571 Mississippi State University Mississippi State, MS 39762 http://www.ece.msstate.edu/~bjones bjones AT ece DOT msstate DOT edu voice 662-325-3149 fax 662-325-2298

Our Master, Jesus Christ, is on his way. He'll show up right on time, his arrival guaranteed by the Blessed and Undisputed Ruler, High King, High God.

• 1 Tim. 6:14b-15 (The Message)

[bnmnetp]

@oscarlevin -- Love the idea of adding the "You have..." I think at one point that had flashed through my brain, but didn't make it to my fingers.

I guess "attempt saved" and showing them feedback are two different things. Right now we pop up a dialog if the attempt is not saved, but maybe positive reinforcement is better than negative alerts.

Maybe instead of a flash each component could have a "You have attempted this question N times." I have a little hesitancy about that because I have sat in the classroom with students and watched them interact and many of them are reluctant to check their answer thinking that somehow they will be punished for too many tries. Data indicates that that would seem to be a small number of students.

[bnmnetp]

OK, I've made good progress on this today:

And inside the assignment:

Notice that I changed the "Score Me" button to something a bit more descriptive.

I think for long assignments having the button at the top and bottom makes sense?

Other feedback??

To @bjones1 point about scanning through the long assignment. For most things, I think it is pretty clear whether they have at least attempted and in many cases gotten the answer correct. For example this parsons problem.

If the answer was wrong it would show with red blocks on the right side.

However -- and this will take more work, Do you think something simple like surrounding the exercise with a wider green border (instead of a thin black border) is sufficient to indicate that the student has worked on the assignment? -- maybe make it a dashed or dotted border for color impaired. I think the dashed stands out a lot better.

[bnmnetp]

Quick update -- I changed the column heading from Progress to My Progress I think that is even clearer.

[pearcej]

@bnmnetp Great work, Brad! Thank you.

'> @bnmnetp "I think for long assignments having the button at the top and bottom makes sense?" Yes, I agree!

'> @bnmnetp "If the answer was wrong it would show with red blocks on the right side. '> However... and this will take more work, Do you think something simple like surrounding the exercise with a wider '> green border (instead of a thin black border) is sufficient to indicate that the student has worked on the assignment? '> -- maybe make it a dashed or dotted border for color impaired. I think the dashed stands out a lot better." Personally, I do, especially if a dashed green border has some consistent meaning such as making progress, but not marked as complete.

[bnmnetp]

I put Score in its own column. I left the column as Not Graded unless the instructor has graded the assignment. If a student clicks through they can still see the "Self Grade" score with all of its caveats. My Progress is still active. I think this makes it clear that the progress and the score are two different things. It should send the message that the student can continue to work on the assignment even if it has been graded by the instructor. I think this solves @bhoffman0 's issue.

I don't think putting the unofficial score here is a good idea as it reinforces the unofficial nature of the self score... But I could be convinced otherwise.

I'm also thinking of putting deadlines that are past in red ? Maybe I should only do that if the the progress is not complete? This would help students see work that is not complete and overdue.

[Alex-Jordan]

I'm reading down the thread writing thoughts as I go.

Runestone: (grades, scores, checks, evaluates)? I know you may ask why not assign points in step 3?

"Points" are a feature of an assignment. But something like a score (on a scale of 0 to 1) could be an abstract assessment for that exercise. If its score is 0.5, and in one context there are 2 possible points and another context 7, why not compute 1 and 3.5 on the fly? The instructor could even change their weighting and make that 7-point exercise be worth 8 points. And automatically the student's point count goes up to 4.

I'd like to hear...

Something I have considered for WeBWorK is a visual representation of progress that goes beyond words and numbers. You've got three states for an individual exercise:

1. attempted, a score was already assessed (either automatically or manually)
2. attempted, a score needs to be assessed manually but that has not yet happened (this may include retries on a manually assessed exercise)
3. not yet attempted (nothing has been typed or clicked on)

Now for an assignment, present a bar with three segments, where the first segment is subdivided for percent correct. It might look like this, except someone who is not colorblind like I am would choose more pleasing colors:

Can you see that I have not yet attempted about 40% of the assignment? I might have as much as 50% of the assignment correct depending on what happens with the instructor's manual scoring. There's a small amount that I tried already and was assessed as wrong.

A couple of concrete things I would like to try in the short term

I like all these things. And Oscar's rewording.

flash a "attempt saved"

I would only like this if it is not a full dialog window popup. If it were not obtrusive, sort of like David's pink flash. Some message appears somewhere near the problem, and fades away. Some attention would be needed to accessibility to make sure AT users get the message. But that might be as easy as having the submit button label be clear.

Runestone eliminates the concept of assignment submission. This is foreign to most students and instructors.

That's not my experience. It may be a culture difference between math and CS, or between the level of math I most often teach (basic algebra) and higher level stuff. But the platforms I am familiar with have long ago dispensed with a final submission button. (The exception is a quiz, where you don't normally expect to give students feedback in between exercises. Those still have a final submission button.)

OK, I've made good progress on this today:

Very nice.

What about removing the "Not Started" text and making that first option be something null? It's confusing to the uninitiated to come to an assignment that you worked on yesterday, and the system is telling you it's "not started". Yes, that's because you never changed the setting. But it will eternally take students being trained about that. If it is something like null, it's still a 3rd option you can reset to, but it doesn't unintentionally say something that is false. (That your assignment that you started is "not started".)

At the other end, the option "Complete" sounds like another of those terms that can mean different things to different people, or even to the same person in a different state of mind. Is there an alternative like "Ready for Review"? So the student can say I'm done with this set, even if they are aware that it is incomplete.

I think for long assignments having the button at the top and bottom makes sense?

Is there an easy way to make it sticky to the top of the page as you scroll? Or would that be undesirable?

If the answer was wrong it would show with red blocks on the right side. Do you think something simple like surrounding the exercise with a wider green border ... maybe make it a dashed or dotted border

That's a lot of important information for the student to put into CSS styling. A blind AT user won't get those signals. So while I think all of this is good to have, would there also be something in the plain HTML that delivers the same information?

I like the changes so far!

[bnmnetp]

@Alex-Jordan I like the idea of a placeholder at the top of the menu.

I think that really shows that tracking progress is optional and something that the student must do. In the assignment itself, the messages for Not Started and null are the same. Click this button to mark in progress....

I don't understand the objection to the word complete. What are the different meanings that you are expecting? Ready for review sounds like a PR or an essay, and again we do not want to ascribe a meaning like submit.

[oscarlevin]

Love the "track my progress" initial state. Or "your"?

What about "I'm Done" instead of complete?

[pearcej]

I like "Track my Progress" and "I'm done". Both make it clearer that this is a tool for "me" (the student.)

[bnmnetp]

I changed the non-option to "Track my Progress"

Changing to I'm done will require a bit more work as Complete is already used in many places in the code and is in the database.

"I'm done" is a bit trickier than "I am Done" because of that darn single quote.

So before heading down that path I would like to be convinced that "I'm Done" is significantly better than Complete.

[bnmnetp]

For progress what about using a simple set of glyphs? This avoids the color blind problems and makes it pretty obvious, especially if you pick good glyphs, which I've spent all of 2 minutes doing here.

✅✅✅✅❌❌🚧🔲🔲🔲

Some of us have had a discussion about using some of these glyphs on the questions themselves to indicate their state at a glance.

Edit: - I would add that I think this can be thought of as a longer term project. I think students go to the chooseAssignment page because they have a very specific goal in mind -- "What do I need to work on next??" Giving them fancy information about status and grading and scoring seems like overkill.

This might fit better in some kind of "student dashboard" to help them better understand their progress in the course.

[Alex-Jordan]

objection to the word complete.

To be sure, I think most of the time it will be understood as intended. But I imagine two issues with it in the mind of a student.

1. "I'm confused because it's not my place to decide if this assignment is "complete" or not. That is the instructor's decision to make."
2. "I know I'm only 3/4 through this assignment, but I'm giving up on the rest of it because I've got too much else to do. It's not actually 'complete' so what should I mark it as?"

[bnmnetp]

So is done a better word than complete then?

[Alex-Jordan]

set of glyphs

If we are getting into my personal aesthetics, I'd prefer the bars. (Also, that is my plan in the works for WeBWorK, and if it's not a good fit for Runestone, that's OK.) As long as there is some more descriptive title/alt text, and as long as the color contrast ratios are high between adjacent bars, I think this kind of thing would be OK for accessibility.

In WeBWorK, I'd have it generate an svg. I might experiment with a pie instead of a bar (but I predict a bar will be better).

is done a better word than complete

I think so. Not to rule out that there may be something even better...

[bnmnetp]

The colored bars are a real challenge, because unless there is some obvious color association, it is a lot of load on the user to recall what each of those colors actually means.

Here are some ideas from the thesaurus from the serious to the ridiculous.

Complete Done Finished Final Concluded discharged wrapped up sewn up polished off sorted out

[Alex-Jordan]

Put a fork in me?

[bnmnetp]

After thinking about it I kind of like Final followed closely by Finished

[oscarlevin]

My vote would be for Finished. I would prefer Finalized over Final.

[pearcej]

After thinking about it I kind of like Final followed closely by Finished I prefer "Finished" because it more clearly places the locus of control on the student.

[bnmnetp]

Well, if nobody else wants to go for polished off, then I'm thinking Finished is a good solution.

[bnmnetp]

WARNING -- I made the change to Finished BUT if you run your own server you will need to run the following database update after this PR is merged.

update grades set is_submit='Finished' where is_submit='Complete';

[bjones1]

This sounds like a good use case for Alembic -- I could create a revision that does this. Would you like me to do this?

[bnmnetp]

Sure, even though its not a schema change I guess it is a good way to communicate the change, at least if anyone is paying attention to alembic...

[github-actions[bot]]

Stale issue message