Is a moodle question type plugin that compares the similarity between student's answer and teacher's answer key using machine learning (natural language processing), and uses the similarity to auto grade the answer. The automatic grade can be manually overridden by the teacher
There is a problem with cosine similarity, which is it cannot detect synonyms. This is unfortunate, because when comparing two texts, we also have to take synonyms into account. After some testing and comparing with manual assessment, the similarity between auto assessment and manual assessment by teacher has a pretty huge gap. Below is the result of the test with 4 different kind of essay question
As you can see, there is a pretty huge gap between manual and auto assessment. Meaning, the auto grading does not correlate with manual grading.
Because of that, we need some way to take synonyms into account. One of the way we can take is using topic modelling. One of the most popular topic modelling is using Latent Semantic Analysis (LSA). To implement LSA, with need to do a Singular Value Decomposition.
After we perform LSA, the similarity between auto assessment and manual assessment by teacher closing in, and in general, the gap is reduced by alot
And the validity of using LSA in auto grading is supported by several papers, particularly this IEEE paper that shows LSA performs better than LDA (another topic modelling method). And another paper that uses LSA as their method of auto grading is this paper.
There is a problem with cosine similarity, which is it cannot detect synonyms. This is unfortunate, because when comparing two texts, we also have to take synonyms into account. After some testing and comparing with manual assessment, the similarity between auto assessment and manual assessment by teacher has a pretty huge gap. Below is the result of the test with 4 different kind of essay question
As you can see, there is a pretty huge gap between manual and auto assessment. Meaning, the auto grading does not correlate with manual grading. Because of that, we need some way to take synonyms into account. One of the way we can take is using topic modelling. One of the most popular topic modelling is using Latent Semantic Analysis (LSA). To implement LSA, with need to do a Singular Value Decomposition.
After we perform LSA, the similarity between auto assessment and manual assessment by teacher closing in, and in general, the gap is reduced by alot
And the validity of using LSA in auto grading is supported by several papers, particularly this IEEE paper that shows LSA performs better than LDA (another topic modelling method). And another paper that uses LSA as their method of auto grading is this paper.