Highlighting keywords when tutors mark essay questions

[weiqingtoh]

I am working on adding a feature for teachers/tutors to highlight certain keywords when grading essay question submissions. This will make it easy for them to eyeball certain parts of the answers instead of reaching the entire thing. I'm still exploring ways to persist it, either through caching or saving it in the model itself.

Ideally, this should look like an input block for tutors to type in keywords that they are looking for in the questions. Each keyword in the student's submission is highlighted in a different color. Keywords can be added or removed easily as well.

Would like to see if anyone has any other ideas/comments/critique on this as well.

Thanks!

[fonglh]

For consistency in marking, should all the tutors be made to use the same set of keywords?

[weiqingtoh]

Interesting suggestion - however I would prefer that tutors are given the choice to choose their own set of keywords. The question to ask here is: Does this help teachers in grading?

To contextualise, this feature was developed for secondary school teachers delivering courses on Coursemology. My opinion is that the use of the keywords depends on how the teachers/tutors/graders use this feature. Some might look out for keywords of common mistakes, others wish to fewer keywords while teachers might want to see all possible keywords. To provide that flexibility, I'd prefer each grader to have full control over their own set of keywords.

The suggestion still has its merits. In the future, the marking assistant should be a feature for teachers to help grade submissions - this includes marking rubrics. Instead of forcing the same set of keywords, it would be good for the lecturer (or head teacher) to 'suggest important keywords' to tutors.

[fonglh]

In that case, it might be useful to allow tutors to toggle the 'suggested important keywords' on with one click while their own custom keywords are deselected.

This will give them the flexibility of using their own keywords and the option to quickly switch to the 'suggested list' if they want to.

[weiqingtoh]

Update

The feature will be named "Marking assistant". Marking Assistant is provided for text-based answers and is able to highlight keywords. Development is done on my forked branch: latest commit 9ace84c6ec .

Using Marking Assistant

1. Marking assistant will highlight individual words only.
   • To add words, user must type their desired word into the text box and press enter.
   • If user enter multiple words ("apple orange"), only the first word will be highlighted ("apple").
   • The search for highlighted words is not case sensitive. ("Apple", "aPPLe", "applE" will be highlighted)
   • Partial words will not be highlighted. ("grapple", "apples" will not be highlighted)
2. Each word will be highlighted with a different color - currently there are 6 colors in the color palette .
3. The user can "Clear All Highlighted Words" using the "Clear All" button, or remove individual words by clicking on the cross sign.
4. Clicking on Show/Hide will toggle the Marking Assistant.
   • Marking Assistant is collapsed when page loads.

Next Steps

• [ ] Implemented suggested keywords. The lecturer/teacher who is setting the question is able to set a few fixed keywords for other graders to retrieve.
• [ ] Persist user keywords. It will be good to remember the user's keywords last used keywords. Currently done through javascript's cache but still has bugs.
• [ ] Button to add marks per keyword. Grader is able to click this button to increase the suggested grade by x marks based on number of highlighted keywords. x can be any number.
• [ ] Think about whether there is a need to separate marking assistant for each question part.

Questions

• Would it be better to cache the last saved user keywords on server or client side? Can anyone point me to how can I do it on the client side? Currently I only use $.cache but I'm not too sure how it works.
• Any comments on the features so far?
• Any suggestions on other features?

[weiqingtoh]

@raymondtangsc @lowjoel @wangqiang1208 - I have a question regarding the storing of these keywords in the models (more of database design). I’ll explain the situation and 2 possible options that I came up with. I’d like to hear your opinions on it as well - really sorry if this is a noob question!

Requirements

As above, I’m building a feature to assist graders - highlight keywords in text-based missions. Currently I have implemented the front end of it, and am looking to implement the models. There are two key requirements:

• Suggested Keywords: Each Assessment/Question will have a set of suggested keywords set by the mission setter
  • Any grader is able to click on the ’Suggested Keywords’ button, and retrieve this information immediately.
• Persisted Keywords: The set of keywords should persist as graders mark different students
  • Each set of keywords should be unique to the grader and assessment

Options

Option 1 - Two separate keyword tables Definitely not the best, but in view that suggested keywords might not match persisted keywords, it could be that we can separate suggested keywords as part of 1 table, and persisted keywords as part of another table. Option 2 - One keyword table with joins We can create a master keyword table, with all the different keywords (suggested and persisted). For each assessment, we can create a suggested_keyword association, and for each user_course_assessment, we can create a persisted_keyword association.

Considerations

I plan to update the set of persisted keywords via AJAX calls upon every add/remove of a keyword => does this make sense?

Regarding keyword models, by right we should be normalising the tables and go for option 2. I’m just wondering if this implementing will be more efficient, and why?

My guess is that it depends on how many common words there are in suggested and persisted keywords. If the intersection is large, then we save space moving to option 2. In terms of search time I think both are similar. Option 1 is only useful as it might be slightly easier to implement.

cc- @fonglh @llja0112 @jsyeo @kxmbrian : Feel free to add on details or points as well.